by Luc Saffre · in Developers · assigned to Sharif Mehedi

Résumé

Concepts sous-jacents

Commentaires

• 4 ans, 2 mois ago by Sharif Mehedi about #4217 (Run schedule.run_pending() using multiprocessing) : My primary work is done on LinodConsumer.
• 4 ans, 2 mois ago by Sharif Mehedi in reply to Sharif Mehedi about #4217 (Run schedule.run_pending() using multiprocessing) : For the moment I am writing a manage.py command to do it, later it will probably be up to you to decide where to run this command from.
• 4 ans, 2 mois ago by Luc Saffre in reply to Sharif Mehedi about #4217 (Run schedule.run_pending() using multiprocessing) : That supervisor config should be created by getlino if you ask me.
• 4 ans, 2 mois ago by Sharif Mehedi about #4217 (Run schedule.run_pending() using multiprocessing) (2 replies) : I run linod channel on my local supervisor and created a template which should be rendered and put to /etc/supervisor/conf.d/ directory. I am not sure where to put the code to do this, may on the pm prep command. @luc , if you have something else on your mind please let me know.
• 4 ans, 2 mois ago by Sharif Mehedi about #4217 (Run schedule.run_pending() using multiprocessing) : I converted the schedule_often and schedule_daily to co-routines; they work fine when I run them on BASH but I am having trouble running the runworker command in supervisor. I probably need to spend more time on that to get around the problems.
• 4 ans, 2 mois ago by Luc Saffre about #4217 (Run schedule.run_pending() using multiprocessing) : I think we will also get rid of the schedule Anyway sackage. Such "scheduled routines" make sense only on a production site where channels is being used.
• 4 ans, 2 mois ago by Sharif Mehedi about #4217 (Run schedule.run_pending() using multiprocessing) : To move the Message.emit_notification to websocket protocol we need to rethink the whole implementation of it. On the other hand, at the moment send_notification is delegating desktop notification to channels when use_websockets is True (which is right for our use case) but when we set use_push_api ...
• 4 ans, 2 mois ago by Sharif Mehedi about #4217 (Run schedule.run_pending() using multiprocessing) : There is a substitute for multiprocessing (and/or subprocess) in django-channels which is `runworker` command. runworker will still be a seperate process as linod but if we decide to use runworker and AsyncConsumer on linod, linod should become much more memory + CPU efficient.
• 4 ans, 2 mois ago by Sharif Mehedi about #4217 (Run schedule.run_pending() using multiprocessing) : Since a subprocess from multiprocessing forks (or copies) the parent process it should do logging just as the parent process does.
• 4 ans, 2 mois ago by Sharif Mehedi about #4217 (Run schedule.run_pending() using multiprocessing) : Multiprocessing takes up more memory then asyncio future. What multiprocessing does is that each subprocess (from multiprocess module) makes a fork (copy the whole thing) of the parent process, which would take more memory as the number of subprocess grows. On the other hand, asyncio future runs on ...
• 5 ans, 4 mois ago by Luc Saffre about #4217 (Run schedule.run_pending() using multiprocessing) : A the moment getlino installs two supervisor tasks for each production site, which causes complexity and limits the number of demo sites on a server with 2GB RAM.
• 5 ans, 4 mois ago by Luc Saffre about #4217 (Run schedule.run_pending() using multiprocessing) : This might avoid us to run two separate Lino processes on a production server.