by Luc Saffre · in Developers · assigned to Luc Saffre

Résumé

Concepts sous-jacents

• #5173 (Lots of useless messages about glyphs in lino.log) (by Luc Saffre in Developers assigned to Luc Saffre)
• #5181 (checkdata says "qty (-36:24 != -36:24)") (by Luc Saffre in Developers assigned to Luc Saffre)
• #5190 (Do we need a new logging level (NOTE)?) (by Luc Saffre in Developers assigned to Luc Saffre)

Commentaires

• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : I think we can consider this as done now.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : It seems that the problems of Jane have vanished over night. Maybe yesterday I had just forgotten to restart supervisor.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : And it seems that email notifications aren't being sent.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : There is still at least one problem on Jane: Seems that background tasks are either not being run, or their running doesn't get logged. When I set the log level of send_emails_often to INFO, there should be a log message every 10 seconds in the lino.log, but nothing appears. To be observed.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : Jane is back.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : Okay, now I am going to upgrade Jane.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : I found yet another series of subtle things to optimize. The default value for the log_level of the procedures.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : One more idea: a virtual field on BackgroundTask that shows the next scheduled execution time.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : And I renamed "lino_runworker" to "linod", and we now say "background tasks" instead of background jobs" (after ssearching for "task vs job").
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : I now replaced "logger = getLogger(__name__)" by "from lino import logger" in most places.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : Jane is back and logging works again. But there are minor issues, e.g. "Run now" on a background job doesn't seem to do anything.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : I released my changes to pypi and will soon upgrade Jane.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : Seems that now I got it fixed at least on my machine. I use a new function is_logserver(), similar to is_devserver() to determine whether the process should become a log server or not.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : Seems that by removing the receiving end of the log server I actually broke things. For now I renamed the "log" directory so that Jane now works with logging disabled.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) (1 reply) : The problem is that almost nothing gets written to the lino.log file.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : But the background jobs seem to run nicely, and via Config -- >System --> Background jobs I can see the output of the latest run. For example checkdata is interesting.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : Jane is back, but logging does not yet work correctly. To be observed.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : I will release to PyPI now and then upgrade Jane in order to test my work on the field.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : A last failure is in docs/writedocs/makedocs.rst :
  047 >>> lino.startup('lino_book.projects.min1.settings') Expected nothing Got:    Started -c (using lino_book.projects.min1.settings) --> PID 97399
  It also comes only when the whole suite is being run, not when runindividually. The "Started" messag...
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : The failures cam because pytest runs multiple tests in a same process, causing the root logger to be already configured.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) (1 reply) : I removed the content of lino.core.logging module (moving its content to other places). Because "import logging" can accidentally import this module when cwd is lino.core.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : The test suite is still having strange failures that don't occur when I run the individual test. Needs more observation.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : I added a test case that would have detected the bug I (accidentally) discovered yesterday.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : Once more i have the feeling that we should rename lino_runworker to linod. I already removed the old linod (in lino.management.commands) because you can get that list now by saying "pm show linod.JobRules". Renaming this command would mean to change supervisor config files on production sites when ...
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : I did some manual tests and found a flagrant typo bug and did some optimizations.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : The test suites are passing, and before my flight back to Tallinn I will commit my changes even though they are not yet well documented.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : The welfare test suite has 3 failures (all trivial).
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : I fixed the 13 tests theoretically. Now a bed session after starting test suite again.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : Only 13 test failures, most of which are trivial, but at least one of them is very mysterious. That's for tomorrow.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) (1 reply) : The LINO_LOGLEVEL envvar has become important when running the test suite. I added 'pytest-env' to the extras_require argument in setup_info.py, and a line "env: LINO_LOGLEVEL=INFO" to the pytest.ini file.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : IOW I am back in #4595
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : At some moment my changes had caused the log server to log recursively into the lino logger. Sharif helped me to understand this. Now I am meditating about a new "strategy" for starting and using the log server...
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : I currently having some stupid problem in my local changes around async integration. I read [url docs.python.org/3/library/asyncio-task.html this] and [url channels.readthedocs.io/en/latest/introduction.html here] to refresh my memory.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : I realized that we don't need two distinc models JobRule and Job.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : I am "almost done". Now I am wondering why end_emails_often doesn't say anything even when I set loglevel to DEBUG.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : I am experimenting (locally) with a StringLogger class. Looks promising. But I am having at least one unexplained show stopper: "Run now" on a daily job rule does not store the getvalue() into the message.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : After meditating and reading [url docs.djangoproject.com/en/4.2/topics/logging/ this] and [url reinout.vanrees.org/weblog/2017/03/08/logging-verbosity-managment-commands.html this] and [url opensourcehacker.com/2011/02/23/temporarily-capturing-python-logging-output-to-a-string-buffer/ this], I reali...
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : To be meditated. If log_level isn't thread-safe, then I plan to replace the "silent" checkbox by an integer field "verbosity" with choices 0 to 3.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : Maybe I will return to the JobRule.log_level field. But what is the relation between log_level and verbosity? And can I change the log level of the logger at runtime? And is this thread-safe?
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : The test suite now probably passes (except for a failure in lino_book/projects/noi1e/tests/test_google.py which is for @8lurry) but I run it a last time before pushing my changes.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : The test suite has only a few trivial failures. I plan to push my changes soon, when it passes. And then maybe release them to froinde in order to test whether it still works on a production server (which is far from being sure because the use_linod is not really covered by the test suite).
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : I did quite a bunch of aggressive code modifcations, and everything still seems to work. Now I started the test suite locally...
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : I am maybe almost there, maybe still far away. I also went into the consumers module and maybe got lost...
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : I think I will now rewrite parts of the linod plugin: the tasks.py becomes a data checker, and the JobRule.run() will no longer create a new Job instance for each run.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : While trying to understand this issue I started to review some parts of the linod code. For example verbose names of models. And JobRule.naqme is not needed. And JobRule.procedure is now unique.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : I wrote a summary of my observations to Johanna.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : I restarted the server.
• 3 ans ago by Luc Saffre about #5172 (lino_runworker is running but doesn't work) : Created new Ticket #5172 (lino_runworker is running but doesn't work).