the random c.sentto == 2 magic numbers in the code is... pretty
unmaintainable and unless you were aware of who "2" was, it's hard to
know what's going on.
in addition, we force modmail to go through the modmail path instead of
letting users bypass validation checks.
because themotte is ~~webscale~~ async, there is some window of time
where code in the sanitize fn will cause the entire worker to crash and
not in a fun way. anyway this uses gevent to handle timeouts instead of
a signal based system which is non-portable and is very fragile,
especially if themotte adds more async functionality in (something that
would probably further improve the performance of the site)
essentially... we don't want the failure of one request taking down the
entire worker!
Against a clean seeded DB, reduces `GET /post/1/` from 63 queries to
26 by removing redundancies and slow lazy-loaded queries during
top comment pagination.
Also applies eager loading to /viewmore/ with the expected reduction
from 5*(N comments) queries to ~12/request.
For testing locally, use a newly seeded DB to ensure
Comment.descendant_count is populated.
Ref: #485
Fixes#483. Implement a Jinja template filter to shuffle a sequence
(which surprisingly doesn't already exist) and shuffle the relevant
lists in the sidebar.
* fix 401-302-401-302-429 loop
* don't logout users on bad form key, just treat the request as unauthenticated
* Handle None/empty case in validate_formkey.
A supplied empty formkey, or the lack of a supplied formkey (None) is not a valid formkey. Handle this inside the function rather than at the call-site.
* Validate as false if no hashstr or string
Co-authored-by: Snakes <104547575+TLSM@users.noreply.github.com>
Ported in from upstream with adjustments for TheMotte, most notably
universal default to 'new' and fixes to 'hot'. Lumped into this PR
because eager comment loading uses it.
Ported in logic from upstream to use SQLAlchemy eager loading instead
of repeated queries when building a submission_listing. Adjusted
loaded relationships to include only those used on TheMotte.
Using test data from seed_db, before and after:
GET /
|----------|--------|--------|--------|--------|--------|------------|
| Database | SELECT | INSERT | UPDATE | DELETE | Totals | Duplicates |
|----------|--------|--------|--------|--------|--------|------------|
| default | 83 | 0 | 0 | 0 | 83 | 72 |
|----------|--------|--------|--------|--------|--------|------------|
Total queries: 83 in 0.031s
GET /
|----------|--------|--------|--------|--------|--------|------------|
| Database | SELECT | INSERT | UPDATE | DELETE | Totals | Duplicates |
|----------|--------|--------|--------|--------|--------|------------|
| default | 14 | 0 | 0 | 0 | 14 | 0 |
|----------|--------|--------|--------|--------|--------|------------|
Total queries: 14 in 0.00718s
Generally standardizes the get_* helpers:
- Adds type hinting.
- Deduplicates block property addition.
- Respects `graceful` in more contexts.
- More resilient to invalid user input / less boilerplate necessary
at call-sites.
* titles: use rdrama's title finding code
this fixes a potential DoS in some really weird pages (seems to be a bug with BS4)
we're not parsing arbitrary HTML
in addition we make some nice checks
* unescape title to fix bug from upstream
* fix nameerror
* Do not proxy requests, since no proxy available.
On the upstream, the `proxies` dict was intended to use a local SOCKS
proxy running on port 18080 with the express purpose of masking the
server IP address. TheMotte isn't running behind a reverse proxy, so
this purpose is moot. Additionally, we don't have a proxy running in
Docker nor do we appear to have one on prod, which breaks autotitle
and thumbnailing regardless--not sure it matters for TheMotte's
use case, but both codepaths have been inoperative because of it.
* use gevent to timeout the function to prevent a
second theoretical DoS by sending data rly slowly
ref: 816389cf28
Co-authored-by: TLSM <duolsm@outlook.com>
