How to create date-wise celery log file?

Question

I have created django website in which I am using celery service for task scheduling.

Current problem is that everyday celery log file increasing & within 3-4 days log file size reach upto 100 mb size.

I have created code so that celery service should able to create log file date-wise but next day celery inserting log into same previous date file.

def SvcDoRun(self):
        logging.info('Starting {name} service ...'.format(name=self._svc_name_))
        os.chdir(INSTDIR) # so that proj worker can be found
        logging.info('cwd: ' + os.getcwd())
        self.ReportServiceStatus(win32service.SERVICE_RUNNING)
        command = '"{celery_path}" -A {proj_dir} worker --pool=solo -f "{log_path}" -l info'.format(
            celery_path=os.path.join(PYTHONSCRIPTPATH, 'celery.exe'),
            proj_dir=PROJECTDIR,
            log_path=os.path.join(INSTDIR,'celery_'+cur_date+'.log'))
        logging.info('command: ' + command)
        args = shlex.split(command)
        proc = subprocess.Popen(args)
        logging.info('pid: {pid}'.format(pid=proc.pid))
        self.timeout = 3000
        while True:
            rc = win32event.WaitForSingleObject(self.hWaitStop, self.timeout)
            if rc == win32event.WAIT_OBJECT_0:
                # stop signal encountered
                # terminate process 'proc'
                PROCESS_TERMINATE = 1
                handle = win32api.OpenProcess(PROCESS_TERMINATE, False, proc.pid)
                win32api.TerminateProcess(handle, -1)
                win32api.CloseHandle(handle)
                break

please help to overcome above problem & let me know if anyone have better solution to create separate log file for each date.

Thanks.

Answer 1

Obvious solution is a daily rotation of your logs, consider using logrotate