Loading...
 
ESA > Join & Share > Forums > Toolbox Forum > Automatic log cleaning or reduced log verbosity

Toolbox Forum

Help

Show posts:
Jump to forum:

Automatic log cleaning or reduced log verbosity

Hello,
We have discovered an issue with the TOOLBOX (version 7.0) logging system that prevents new operations to be accepted and processed correctly when a large number of operations is logged.

After some investigation the issue is traced to the file system limits in sub directories allocation: when an operation is executed by the toolbox, an extended log is placed in a sub directory of the main service log directory. After 31999 operations (that is the ext3 limit), the next operations are rejected with a "File not found" error upon creation of the log subdirectory.

Presently we solved the issue by adding a cron task that removes older logs but a good solution should lie within the TOOLBOX itself, here are two suggestions for that:

1 - Automatic log cleaning:
Per service setting (with toolbox wide default) of the maximum age and number of execution logs (upon receiving a new operation, the TOOLBOX should check the log dir and remove logs according to their age and number).

2 - Log verbosity:
For a development installation the log level is appropriate but there should be a per-service configuration option allowing to reduce the log verbosity for production services; i.e. for operations completed without error the log subdirectory should be entirely removed upon operation completion (maybe leaving just a record in a main log file) in a production service and only unsuccessful operations' log should be kept.

Best regards,
Alan


Re: Automatic log cleaning or reduced log verbosity

Hello Alan,

starting from version 8.0, Toolbox source code has been made publicly available and has been published on Google Code (see http://code.google.com/p/toolboxenvironment/). On the project issue tracker i have added an enhancement request (#28) which covers your above request. I suggest you to take a look at the issue tracker and submit suggestions (if any).
Kind regards,
TOOLBOX Support Team


> Hello,
> We have discovered an issue with the TOOLBOX (version 7.0) logging system that prevents new operations to be accepted and processed correctly when a large number of operations is logged.
>
> After some investigation the issue is traced to the file system limits in sub directories allocation: when an operation is executed by the toolbox, an extended log is placed in a sub directory of the main service log directory. After 31999 operations (that is the ext3 limit), the next operations are rejected with a "File not found" error upon creation of the log subdirectory.
>
> Presently we solved the issue by adding a cron task that removes older logs but a good solution should lie within the TOOLBOX itself, here are two suggestions for that:
>
> 1 - Automatic log cleaning:
> Per service setting (with toolbox wide default) of the maximum age and number of execution logs (upon receiving a new operation, the TOOLBOX should check the log dir and remove logs according to their age and number).
>
> 2 - Log verbosity:
> For a development installation the log level is appropriate but there should be a per-service configuration option allowing to reduce the log verbosity for production services; i.e. for operations completed without error the log subdirectory should be entirely removed upon operation completion (maybe leaving just a record in a main log file) in a production service and only unsuccessful operations' log should be kept.
>
> Best regards,
> Alan



Re: Automatic log cleaning or reduced log verbosity

Hello Alan,

thanks for reporting the problem. We probably implement a solution in the Revision 3 of Toolbox 8.0. I will let you know asap. I take the opportunity to suggest you all to follow the Twitter account toolboxintecs. This will provide info and links in the near future.
Regards,
Toolbox Support

> Hello,
> We have discovered an issue with the TOOLBOX (version 7.0) logging system that prevents new operations to be accepted and processed correctly when a large number of operations is logged.
>
> After some investigation the issue is traced to the file system limits in sub directories allocation: when an operation is executed by the toolbox, an extended log is placed in a sub directory of the main service log directory. After 31999 operations (that is the ext3 limit), the next operations are rejected with a "File not found" error upon creation of the log subdirectory.
>
> Presently we solved the issue by adding a cron task that removes older logs but a good solution should lie within the TOOLBOX itself, here are two suggestions for that:
>
> 1 - Automatic log cleaning:
> Per service setting (with toolbox wide default) of the maximum age and number of execution logs (upon receiving a new operation, the TOOLBOX should check the log dir and remove logs according to their age and number).
>
> 2 - Log verbosity:
> For a development installation the log level is appropriate but there should be a per-service configuration option allowing to reduce the log verbosity for production services; i.e. for operations completed without error the log subdirectory should be entirely removed upon operation completion (maybe leaving just a record in a main log file) in a production service and only unsuccessful operations' log should be kept.
>
> Best regards,
> Alan



Show posts:
Jump to forum: