From be952c2771e822b0c56e9f9e679f42ec2732df91 Mon Sep 17 00:00:00 2001 From: justcool393 Date: Wed, 29 Mar 2023 14:32:48 -0700 Subject: [PATCH] post scheduling (#554) * prepare codebase to create scheduled tasks there is some prep work involved with this. the scheduler would be happy if this work was done. simply, we extract out the `created_utc` interface from *everything* that uses it such that we don't have to repeat ourselves a bunch. all fun stuff. next commit is the meat of it. * cron: basic backend work for scheduler * avoid ipmort loop * attempt 2 at fixing import loops * parathensize because operator precedence * delete file that came back for some reason. * does NOPing the oauth apps work? * import late and undo clients.py change * stringify column names. * reorder imports. * remove task reference * fix missing mapper object * make coupled to repeatabletask i guess * sanitize: fix sanitize imports * import shadowing crap * re-shadow shadowed variable * fix regexes * use the correct not operator * readd missing commit * scheduler: SQLA only allows concrete relations * implement submission scheduler * fix import loop with db_session * get rid of import loop in submission.py and comment.py * remove import loops by deferring import until function clal * i give up. * awful. * ... * fix another app import loop * fix missing import in route handler * fix import error in wrappers.py * fix wrapper error * call update wrapper in the admin_level_required case * :marseyshrug: * fix issue with wrapper * some cleanup and some fixes * some more cleanup let's avoid polluting scopes where we can. * ... * add SCHEDULED_POSTS permission. * move const.py into config like the other files. * style fixes. * lock table for concurrency improvements * don't attempt to commit on errors * Refactor code, create `TaskRunContext`, create python callable task type. * use import contextlib * testing stuff i guess. * handle repeatable tasks properly. * Attempt another fix at fighting the mapper * do it right ig * SQLA1.4 doesn't support nested polymorphism ig * fix errenous class import * fix mapper errors * import app in wrappers.py * fix import failures and stuff like that. * embed and import fixes * minor formatting changes. * Add running state enum and don't attempt to check for currently running tasks. * isort * documentation, style, and commit after each task. * Add completion time and more docs, rename, etc * document `CRON_SLEEP_SECONDS` better. * add note about making LiteralString * filter out tasks that have been run in the future * reference RepeatableTask's `__tablename__` directly * use a master/slave configuration for tasks the master periodically checks to see if the slave is alive, healthy, and not taking too many resources, and if applicable kills its child and restarts it. only one relation is supported at the moment. * don't duplicate process unnecessarily * note impl detail, add comments * fix imports. * getting imports to stop being stupid. * environment notes. * syntax derp * *sigh* * stupid environment stuff * add UI for submitting a scheduled post * stupid things i need to fix the user class * ... * fix template * add formkey * pass v * add hour and minute field * bleh * remove concrete * the sqlalchemy docs are wrong * fix me being dumb and not understanding error messages * missing author attribute for display * author_name property * it's a property * with_polymorphic i think fixes this * dsfavgnhmjk * *sigh* * okay try this again * try getting rid of the comment section * include -> extends * put the div outside of the thing. * fix user page listings :/ * mhm * i hate this why isn't this working * this should fix it * Fix posts being set as disabled by default * form UI imrpovements * label *
diff --git a/files/templates/admin/tasks/scheduled_post.html b/files/templates/admin/tasks/scheduled_post.html new file mode 100644 index 000000000..9587432bb --- /dev/null +++ b/files/templates/admin/tasks/scheduled_post.html @@ -0,0 +1,22 @@ +{%- extends "submission.html" -%} +{%- block comment_section -%} +
+ You are viewing a single scheduled post. View all scheduled posts + {%- if v.admin_level >= PERMS['SCHEDULER'] %} or the associated task{%- endif -%} +
+{%- if v.admin_level >= PERMS['SCHEDULER'] -%} +
+
Edit Scheduled Post
+
+ {{forms.scheduled_post_time_selection_form(v, p)}} +
+
+{%- endif -%} +
+
Previous Task Runs
+ {%- with listing = p.submissions -%} + {%- include "submission_listing.html" -%} + {%- endwith -%} +
+{%- endblock -%} +{%- block comment_section2 -%}{%- endblock -%} diff --git a/files/templates/admin/tasks/scheduled_posts.html b/files/templates/admin/tasks/scheduled_posts.html new file mode 100644 index 000000000..471caa8c4 --- /dev/null +++ b/files/templates/admin/tasks/scheduled_posts.html @@ -0,0 +1,20 @@ +{%- extends 'default.html' -%} +{%- block content -%} +
+ {%- include "submission_listing.html" -%} +
+
+
Submit a Scheduled Post
+

+ This tool allows you to submit a scheduled post. It will be submitted as you at the scheduled date and time, and will perform tasks as if a post was submitted manually (including notifying any subscribers and any post-submit actions).
+ You may use Python strftime formatting in the title of your submitted post if you want to include the date in your submitted post. +

+
+ {# (label, name, id, min_length=none, max_length=none, required=false) #} + {{forms.text_field("Title", "title", "title", min_length=0, max_length=SUBMISSION_TITLE_LENGTH_MAXIMUM, required=true)}} + {{forms.text_field("URL (optional)", "url", "url", min_length=0, max_length=2048, required=false)}} + {{forms.textarea_field("Body", "body", "body", min_length=0, max_length=SUBMISSION_BODY_LENGTH_MAXIMUM, required=false)}} + {{forms.scheduled_post_time_selection_form(v, none)}} +
+
+{%- endblock -%} diff --git a/files/templates/admin/tasks/single_run.html b/files/templates/admin/tasks/single_run.html new file mode 100644 index 000000000..85cdf374c --- /dev/null +++ b/files/templates/admin/tasks/single_run.html @@ -0,0 +1,41 @@ +{%- extends "default.html" -%} +{%- block content -%} +

Run for Task #{{run.task_id}}

+
+
+ + + + + + + + + + + + + + + {%- if run.completed_utc -%} + + + + + + + + + {%- endif -%} + +
ID{{run.id}}
Status{{run.status_text}}
Started (UTC){{run.created_datetime}} ({{run.age_string}})
Completed (UTC){{run.completed_datetime_str}} ({{run.completed_utc | agestamp}})
Elapsed Time{{run.time_elapsed_str}}
+
+
+{%- if run.traceback_str and v.admin_level >= PERMS['SCHEDULER_TASK_TRACEBACK'] -%} +
+

Exception Traceback

+

During this run, the task encountered an unhandled exception.

+ 
{{run.traceback_str}}
+
+{%- endif -%} +{%- endblock -%} diff --git a/files/templates/admin/tasks/single_task.html b/files/templates/admin/tasks/single_task.html new file mode 100644 index 000000000..6c9f0f3e0 --- /dev/null +++ b/files/templates/admin/tasks/single_task.html @@ -0,0 +1,91 @@ +{%- extends "default.html" -%} +{%- block content -%} +

Task #{{task.id}}

+
+ You are viewing a single task. View all tasks +
+
+
+ + + + + + + + + + + + + + + + + + + + + + + {%- if task.type == ScheduledTaskType.SCHEDULED_SUBMISSION -%} + + + + + {%- elif task.type == ScheduledTaskType.PYTHON_CALLABLE -%} + + + + + + + + + {%- endif -%} + + + + + +
ID{{task.id}}
Status{{task.run_state_enum.name.title()}}
Created (UTC){{task.created_datetime}} ({{task.age_string}})
Last Run (UTC){{task.run_time_last_str}}
Next Run (UTC){{task.next_trigger(task.run_time_last_or_created_utc) | timestamp}}
Scheduled Post + {%- if v.admin_level >= PERMS['SCHEDULER_POSTS'] -%} + {{task.title}} + {%- else -%} + {{task.title}} + {%- endif -%} +
Import Path{{task.import_path}}
Callable{{task.callable}}
Enabled{{task.enabled}}
+
+
+
+
Edit Scheduled Task
+
+ {{forms.scheduled_post_time_selection_form(v, task)}} +
+
+
+
Previous Task Runs
+
+ + + + + + + + + + + {%- for run in task.runs -%} + + + + + + + {%- endfor -%} + +
IDStatusStartedCompleted
{{run.id}}{{run.status_text}}{{run.age_string}}{{run.completed_utc | agestamp}}
+
+
+{%- endblock -%} diff --git a/files/templates/admin/tasks/tasks.html b/files/templates/admin/tasks/tasks.html new file mode 100644 index 000000000..314254bf0 --- /dev/null +++ b/files/templates/admin/tasks/tasks.html @@ -0,0 +1,30 @@ +{%- extends "default.html" -%} +{%- block content -%} +

Scheduled Tasks

+
+
+ + + + + + + + + + + + {%- for task in listing -%} + + + + + + + + {%- endfor -%} + +
TaskCreatedTypeStatusEnabled
Task {{task.id}}{{task.age_string}}{{task.type}}{{task.run_state_enum.name.title()}}{{task.enabled}}
+
+
+{%- endblock -%} diff --git a/files/templates/chat.html b/files/templates/chat.html index 1930226b7..0082083ff 100644 --- a/files/templates/chat.html +++ b/files/templates/chat.html @@ -152,7 +152,7 @@ {{m['username']}} {% if not same %} - {{m['time'] | timestamp}} + {{m['time'] | agestamp}} {% endif %}
diff --git a/files/templates/component/comment/voting_desktop.html b/files/templates/component/comment/voting_desktop.html index 11bb0d499..8edf0f392 100644 --- a/files/templates/component/comment/voting_desktop.html +++ b/files/templates/component/comment/voting_desktop.html @@ -19,7 +19,7 @@ {% if voted == -1 %}
  • {% endif %} -{%- elif environ.get('DISABLE_DOWNVOTES') == '1' -%} +{%- elif not ENABLE_DOWNVOTES -%} {# downvotes not allowed, nop #} {%- elif v -%} diff --git a/files/templates/component/comment/voting_mobile.html b/files/templates/component/comment/voting_mobile.html index 685ccae6e..caaf74334 100644 --- a/files/templates/component/comment/voting_mobile.html +++ b/files/templates/component/comment/voting_mobile.html @@ -13,7 +13,7 @@
  • {{score}} - +
    • {% else %}
  • diff --git a/files/templates/component/modal/award.html b/files/templates/component/modal/award.html index 2b5e29702..6eac849e1 100644 --- a/files/templates/component/modal/award.html +++ b/files/templates/component/modal/award.html @@ -12,7 +12,7 @@
    {% for award in v.user_awards %} - +
    {{award.title}}
    {{award.owned}} owned
    diff --git a/files/templates/component/post/actions.html b/files/templates/component/post/actions.html index 5bbe3d05d..6d5891d8a 100644 --- a/files/templates/component/post/actions.html +++ b/files/templates/component/post/actions.html @@ -1,3 +1,5 @@ +{%- if p.is_real_submission -%} + {% if v and v.id==p.author_id and p.private %}
    @@ -100,3 +102,4 @@ {% endif %} {% endif %} +{%- endif -%} {# {%- if p.is_real_submission -%} #} \ No newline at end of file diff --git a/files/templates/component/post/actions_mobile.html b/files/templates/component/post/actions_mobile.html index 261d8a6a8..978508357 100644 --- a/files/templates/component/post/actions_mobile.html +++ b/files/templates/component/post/actions_mobile.html @@ -1,3 +1,4 @@ +{%- if p.is_real_submission -%} {% if v and v.id==p.author_id and p.private %} @@ -58,3 +59,4 @@ {% endif %} {% endif %} +{%- endif -%} {# {%- if p.is_real_submission -%} #} diff --git a/files/templates/contact.html b/files/templates/contact.html index eeb333673..7ad3480fd 100644 --- a/files/templates/contact.html +++ b/files/templates/contact.html @@ -24,7 +24,7 @@ {% endif %} - + {{forms.formkey(v)}}