Commit Graph

55 Commits (e84e25d8fa43fc57d590000cec3cd556ba129be2)

Author SHA1 Message Date
flamingos-cant 19a1a077c5
Add a blocklist for URLs. (#4515)
* Add a blocklist for URLs.

* Fix SQL format

* Make clippy happy.

* Use regex for URL matching.

* Escape regex chars in URLs.

* Use post for modification.

* Make URL block regex static and remove API routes.

* Add date fields to table and use transaction.

* Use Cache for blocklist.

* Rename check_links + move list to parameters of process_markdown.

* SQL format.

* Format, again.

* Remove println.

* Add API test.

* Set a shorter lifetime for regex in debug mode.

* Add missing macro.

* Update lemmy-js-client

* Update api_test/pnpm-lock.yaml

* Don't break other tests

* Use different URL for test

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Nutomic <me@nutomic.com>
2 months ago
Nutomic 0f414a95d5
Local only community (#4350)
* Add support for local only community (fixes #1576)

* add filters and tests to db views

* dont federate local only community

* test get apub community http

* tests

* more checks

* wip

* api test

* fix tests

* change community.local_only column to visibility enum
(for private communities)

* sql fmt

* rename vars

* clippy

* fix tests

* update lib

* review

* fix js client version

* update client
4 months ago
Nutomic e8a52d3a5c
Rewrite images to use local proxy (#4035)
* Add markdown rule to add rel=nofollow for all links

* Add markdown image rule to add local image proxy (fixes #1036)

* comments

* rewrite markdown image links working

* add comment

* perform markdown image processing in api/apub receivers

* clippy

* add db table to validate proxied links

* rewrite link fields for avatar, banner etc

* sql fmt

* proxy links received over federation

* add config option

* undo post.url rewriting, move http route definition

* add tests

* proxy images through pictrs

* testing

* cleanup request.rs file

* more cleanup (fixes #2611)

* include url content type when sending post over apub (fixes #2611)

* store post url content type in db

* should be media_type

* get rid of cache_remote_thumbnails setting, instead automatically
take thumbnail from federation data if available.

* fix tests

* add setting disable_external_link_previews

* federate post url as image depending on mime type

* change setting again

* machete

* invert

* support custom emoji

* clippy

* update defaults

* add image proxy test, fix test

* fix test

* clippy

* revert accidental changes

* address review

* clippy

* Markdown link rule-dess (#4356)

* Extracting opengraph_data to its own type.

* A few additions for markdown-link-rule.

---------

Co-authored-by: Nutomic <me@nutomic.com>

* fix setting

* use enum for image proxy setting

* fix test configs

* add config backwards compat

* clippy

* machete

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
4 months ago
Nutomic 9e099726e6
Cleanup checks for community actions (fixes #2858, fixes #2868) (#4028)
* Cleanup checks for community actions (fixes #2858, fixes #2868)

* allow restoring deleted community

* review changes

* remove unneeded sql

* remove joins

* change mod log check
7 months ago
Nutomic 291ff19718
Only sanitize strings when generating RSS feeds and emails (fixes #4003) (#4024)
* Only sanitize strings when generating RSS feeds and emails (fixes #4003)

* clippy

* fix test
7 months ago
Nutomic f858d8cbce
Remove explicit auth params (#3946)
* Remove explicit auth params (ref #3725)

Only take auth via header or cookie. This requires a new version
of lemmy-js-client for api tests to pass.

* rework api_crud

* remove remaining auth params, move logic to session middleware

* fmt, fix test

* update js client

* remove auth param from api tests

* Pass auth as header

* add !

* url vars, setHeader

* cleanup

* fmt

* update

* Updating for new lemmy-js-client.

---------

Co-authored-by: Dessalines <tyhou13@gmx.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
8 months ago
Nutomic 71d61138bc
Replace ammonia lib with manual html escaping (fixes #3774) (#3938)
* Replace ammonia lib with manual html escaping (fixes #3774)

* prettier

* clippy

* remove sanitize unit test

* fix tests

* fix schema
9 months ago
Dessalines 56e26fc3d4
Adding a post_view mode. Fixes #3730 (#3731)
* Adding a post_view mode. Fixes #3730

* Fix test.

* Addressing PR comments.

* Adding a post_view mode. Fixes #3730

* Fix test.

* Addressing PR comments.

* Fixing column order.

* Fix default Ok returns.

* Removing return Err(... where feasible.
9 months ago
dullbananas 969f8b2ce9
Replace TypedBuilder with Default in update forms (#3814)
* Update comment.rs

* Update community.rs

* Update local_site.rs

* Update local_site_rate_limit.rs

* Update local_user.rs

* Update person.rs

* Update comment.rs

* Update community.rs

* Update local_site.rs

* Update local_site_rate_limit.rs

* Update local_user.rs

* Update post.rs

* Update private_message.rs

* Update site.rs

* Update post.rs

* Update person.rs

* Update private_message.rs

* Update comment.rs

* Update create.rs

* Update leave_admin.rs

* Update update.rs

* Update remove.rs

* Update add_admin.rs

* Update verify_email.rs

* Update mod.rs

* Update mod.rs

* Update undo_delete.rs

* Update undo_delete.rs

* Update utils.rs

* Update feature.rs

* Update delete.rs

* Update lock.rs

* Update create.rs

* Update approve.rs

* Update update.rs

* Update lock_page.rs

* Update block_user.rs

* Update delete.rs

* Update undo_block_user.rs

* Update collection_remove.rs

* Update post.rs

* Update hide.rs

* Update person.rs

* Update remove.rs

* Update post_view.rs

* Update create.rs

* Update remove.rs

* Update collection_add.rs

* Update community.rs

* Update update.rs

* Update post_aggregates.rs

* Update update.rs

* Update comment.rs

* Update code_migrations.rs

* Update registration_application_view.rs

* Update update.rs

* Update ban_person.rs

* Update community.rs

* Update delete.rs

* Update delete.rs

* Update delete.rs

* Update person_aggregates.rs

* Update save_settings.rs

* Update distinguish.rs

* Update mark_read.rs

* Update site_aggregates.rs

* Update create.rs

* Fix

* rerun ci

* Update comment.rs

* rerun ci

* Update create.rs

* Update create.rs

* Update post_view.rs

* rerun ci

* Update undo_delete.rs

* rerun ci
10 months ago
Nutomic 27be1efb74
Rewrite remaining federation actions, get rid of PerformCrud trait (#3794)
* Rewrite ban actions

* Rewrite delete/remove actions

* Rewrite remove/delete community

* Rewrite report actions

* Rewrite feature/lock post

* Rewrite update community actions

* Rewrite remaining federation actions

* Get rid of PerformCrud trait

* clippy
10 months ago
Nutomic 3471f3533c
Sanitize html (#3708)
* HTML sanitization in apub code

* Sanitize API inputs

* fmt

* Dont allow html a, img tags

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
10 months ago
dullbananas 1d38aad9d3
Make functions work with both connection and pool (#3420)
* a lot

* merge

* Fix stuff broken by merge

* Get rid of repetitive `&mut *context.conn().await?`

* Add blank lines under each line with `conn =`

* Fix style mistakes (partial)

* Revert "Fix style mistakes (partial)"

This reverts commit 48a033b87f.

* Revert "Add blank lines under each line with `conn =`"

This reverts commit 773a6d3beb.

* Revert "Get rid of repetitive `&mut *context.conn().await?`"

This reverts commit d2c6263ea1.

* Use DbConn for CaptchaAnswer methods

* DbConn trait

* Remove more `&mut *`

* Fix stuff

* Re-run CI

* try to make ci start

* fix

* fix

* Fix api_common::utils

* Fix apub::activities::block

* Fix apub::api::resolve_object

* Fix some things

* Revert "Fix some things"

This reverts commit 2bf8574bc8.

* Revert "Fix apub::api::resolve_object"

This reverts commit 3e4059aabb.

* Revert "Fix apub::activities::block"

This reverts commit 3b02389abd.

* Revert "Fix api_common::utils"

This reverts commit 7dc73de613.

* Revert "Revert "Fix api_common::utils""

This reverts commit f740f115e5.

* Revert "Revert "Fix apub::activities::block""

This reverts commit 2ee206af7c.

* Revert "Revert "Fix apub::api::resolve_object""

This reverts commit 96ed8bf2e9.

* Fix fetch_local_site_data

* Fix get_comment_parent_creator

* Remove unused perma deleted text

* Fix routes::feeds

* Fix lib.rs

* Update lib.rs

* rerun ci

* Attempt to create custom GetConn and RunQueryDsl traits

* Start over

* Add GetConn trait

* aaaa

* Revert "aaaa"

This reverts commit acc9ca1aed.

* Revert "Revert "aaaa""

This reverts commit 443a2a00a5.

* still aaaaaaaaaaaaa

* Return to earlier thing

Revert "Add GetConn trait"

This reverts commit ab4e94aea5.

* Try to use DbPool enum

* Revert "Try to use DbPool enum"

This reverts commit e4d1712646.

* DbConn and DbPool enums (db_schema only fails to compile for tests)

* fmt

* Make functions take `&mut DbPool<'_>` and make db_schema tests compile

* Add try_join_with_pool macro and run fix-clippy on more crates

* Fix some errors

* I did it

* Remove function variants that take connection

* rerun ci

* rerun ci

* rerun ci
10 months ago
Nutomic 93225e5ddf
Error enum fixed (#3487)
* Create error type enum

* Replace magic string slices with LemmyErrorTypes

* Remove unused enum

* Add rename snake case to error enum

* Rename functions

* clippy

* Fix merge errors

* Serialize in PascalCase instead of snake_case

* Revert src/lib

* Add serialization tests

* Update translations

* Fix compilation error in test

* Fix another compilation error

* Add code for generating typescript types

* Various fixes to avoid breaking api

* impl From<LemmyErrorType> for LemmyError

* with_lemmy_type

* trigger ci

---------

Co-authored-by: SleeplessOne1917 <abias1122@gmail.com>
10 months ago
Neshura 203e35899e
Add separate Post check for is_valid_body_field (#3263)
* Add separate Post check for is_valid_body_field

* Modify is_valid_body_check for posts only

* Fix check var reinit in validation.rs

* Extra empty line to rerun woodpecker with changes

* Change Option to bool, add false to non-post calls

* Woodpecker trick.. again

* Probable rust_fmt fail fixed

* cargo_clippy changes

* Missing space between = and if

* Remove ; after body length checks
11 months ago
Nutomic 3565ad984a
Remove chatserver (#2919)
* Remove chatserver

* fix clippy

* Remove captchas (fixes #2922)

* fix prettier

* fix api_common build

* cargo fmt
12 months ago
Nutomic a5707328cf
Dont return error in case optional auth is invalid (#2879)
* Dont return error in case optional auth is invalid

* fixes
1 year ago
Dessalines 38d4429ae7
Adding check to description and body length fields. (#2805)
* Adding check to description and body length fields.

- Also making the checks return `LemmyError`
- Fixes #1747

* Address PR comments.

* PR comments 2
1 year ago
Dessalines 63f54a3103
Making the chat server an actor. (#2793)
* Making the chat server an actor.

- Fixes #2778
- #2787

* Forgot to add handlers folder.

* Some cleanup.

* Forgot to remove a comment.

* Address PR comments.

* Using ToString for enum operations.
1 year ago
Nutomic 165b19e75c
Optimize federated language updates to avoid unnecessary db writes (#2786)
* Optimize federated language updates to avoid unnecessary db writes (fixes #2772)

* fix tests

* fix test, rename functions

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
1 year ago
Dessalines 36aaa50644
Organize utils into separate files. Fixes #2295 (#2736)
* Organize utils into separate files. Fixes #2295

* Moving tests.

* Fix test.

* Fix test 2
1 year ago
Dessalines 956d0aa52f
Fixing updated form fields. Fixes #2724 (#2725)
* Fixing updated form fields. Fixes #2724

* Fix import.
1 year ago
Felix Ableitner c6c52ab9cc Add SendActivity trait so that api crates compile in parallel with lemmy_apub 1 year ago
Felix Ableitner 030afbc2e7 Merge websocket crate into api_common 1 year ago
Nutomic 6f3bf4634b
Various pedantic clippy fixes (#2568)
* Various pedantic clippy fixes

* more clippy pedantic fixes

* try to fix ci

* add fix clippy script, use rust 1.65

* fix clippy
2 years ago
Dessalines 5d837780f5
Add diesel_async, get rid of blocking function (#2510)
* Moving settings to Database.

- Moves many settings into the database. Fixes #2285
- Adds a local_site and instance table. Fixes #2365 . Fixes #2368
- Separates SQL update an insert forms, to avoid runtime errors.
- Adds TypedBuilder to all the SQL forms, instead of default.

* Fix weird clippy issue.

* Removing extra lines.

* Some fixes from suggestions.

* Fixing apub tests.

* Using instance creation helper function.

* Move forms to their own line.

* Trying to fix local_site_data, still broken.

* Testing out async

* Testing out async 2

* Fixing federation tests.

* Trying to fix check features 1.

* Starting on adding diesel async. 1/4th done.

* Added async to views and schema.

* Adding some more async

* Compiling now.

* Added diesel async. Fixes #2465

* Running clippy --fix

* Trying to fix cargo test on drone.

* Trying new muslrust.

* Trying a custom dns

* Trying a custom dns 2

* Trying a custom dns 3

* Trying a custom dns 4

* Trying a custom dns 5

* Trying a custom dns 6

* Trying a custom dns 7

* Addressing PR comments.

* Adding check_apub to all verify functions.

* Reverting back drone.

* Fixing merge

* Fix docker images.

* Adding missing discussion_languages.

* Trying to fix federation tests.

* Fix site setup user creation.

* Fix clippy

* Fix clippy 2

* Test api faster

* Try to fix 1

* Try to fix 2

* What are these lines about

* Trying to fix 3

* Moving federation test back to top.

* Remove logging cat.
2 years ago
Dessalines 235cc8b228
Moving settings to Database. (#2492)
* Moving settings to Database.

- Moves many settings into the database. Fixes #2285
- Adds a local_site and instance table. Fixes #2365 . Fixes #2368
- Separates SQL update an insert forms, to avoid runtime errors.
- Adds TypedBuilder to all the SQL forms, instead of default.

* Fix weird clippy issue.

* Removing extra lines.

* Some fixes from suggestions.

* Fixing apub tests.

* Using instance creation helper function.

* Move forms to their own line.

* Trying to fix local_site_data, still broken.

* Fixing federation tests.

* Trying to fix check features 1.

* Addressing PR comments.

* Adding check_apub to all verify functions.
2 years ago
Nutomic 2ef0f8f5f8
implement language tags for site/community in db and api (#2434)
* implement language tags for site/community in db and api

* add api checks for valid languages

* during db migration, update existing users, sites, communities to have all languages enabled

* init new users/communities with site languages (not all languages)

* federate site/community languages

* fix tests

* when updating site languages, limit community languages to this subset

also, when making a new post and subset of user lang, community lang
contains only one item, use that as post lang

* add tests for actor_language db functions

* include language list in siteview/communityview

* Fix some of the review comments

* Some more review changes

* Add todo about boxed query

* Add default_post_language to GetCommunityResponse
2 years ago
Dessalines 8bfeb4b627
Fix a few form options for diesel. Fixes #2287 (#2376)
* Fix a few form options for diesel. Fixes #2287

* Adding TODO comment.
2 years ago
Nutomic a2a594b763
Extract Activitypub logic into separate library (#2288)
* Create example for apub lib

* some rewriting of apub lib

* Add LocalInstance struct for apub lib to avoid using Lemmy Settings

* Move ActorType trait to lemmy_apub, because its not needed in library

* Use reqwest_retry instead of custom impl, dont specify timeout on every send()

* Some improvements to example

* Moved inbox handling to library

* bug fixes

* Move context and serde helpers into library

* wip: example changes

* Add lemmy_utils feature to build only LemmyError

* Rename to activitypub_federation

* Remove lemmy_utils dep from activitypub_federation using generic error type

* Finish activitypub example

* Cleanup and fix tests

* Reorganize library files

* Remove ApubObject.to_tombstone()

* Extract activitypub library into separate git repository
2 years ago
Nutomic f0ec3015cd
Remove check that avatars/banners are locally hosted (fixes #2254) (#2255) 2 years ago
Nutomic 3aa3d75a1e
Add cargo feature for building lemmy_api_common with mininum deps (#2243) 2 years ago
Nutomic dd5835fb6e
Add restricted community field to CreateCommunity, UpdateCommunity (ref #2235) (#2242) 2 years ago
Nutomic 7058cfa1cf
Implement restricted community (only mods can post) (fixes #187) (#2235)
* Implement restricted community (only mods can post) (fixes #187)

* review fixes

* fix tests
2 years ago
Dessalines 37c834725c Make sure application questionaire is required. Fixes #2189
Also fix local image domain check. Was failing for blank strings /
replaces.
2 years ago
Nutomic bcf7ec6109
Forbid remote URLs for avatars/banners (fixes #1618) (#2132) 2 years ago
Nutomic 166ec196b0
Rework error handling (fixes #1714) (#2135) 2 years ago
dayinjing 7f9b55e793
Hide community v2 (#2055)
* Initial working of hiding communities and adding a db entry for mod log

* Return mod log for hidden communities

* Clean up hidding communities PR

* use lower case like other migration files

* Formatting fix

* pass in admin id to list, make match logic the same in post_view as community_view. Dont force non null for reason

* Clean PR review stuff

* Change person_id to mod_person_id on hide community table

* Make bools optional, add a space for formating

Co-authored-by: Thor Odinson <odinson@asgard.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2 years ago
Riley 35cbae61bc
Don't drop error context when adding a message to errors (#1958)
* Respond directly with LemmyError

Instrument Perform implementations for more precise traces
Use ApiError to format JSON errors when messages are present
Keep SpanTrace output in LemmyError Display impl

* Hide SpanTrace debug output from LemmyError

* Don't log when entering spans, only when leaving

* Update actix-web

* Update actix-rt

* Add newline after error info in LemmyError Display impl

* Propogate span information to blocking operations

* Instrument apub functions

* Use skip_all for more instrument attributes, don't skip 'self' in some api actions

* Make message a static string

* Send proper JSON over websocket

* Add 'message' to LemmyError display if present

* Use a quieter root span builder, don't pretty-print logs

* Keep passwords and emails out of logs

* Re-enable logging Login

* Instrument feeds

* Emit our own errors

* Move error log after status code recording

* Make Sensitive generic over the inner type

* Remove line that logged secrets
2 years ago
Dessalines 05d43150bc Making public key required. Fixes #1934 3 years ago
Felix Ableitner c725514841 Change to_apub and from_apub to take by value and avoid cloning 3 years ago
Felix Ableitner 03d8ac75ef Move activity structs to protocol folder 3 years ago
Felix Ableitner 1aa0e1997b Major refactor, adding newtypes for apub crate
- this allows moving FromApub/ToApub traits into apub lib
3 years ago
Felix Ableitner f24999027e Merge crates db_schema and db_queries 3 years ago
Nutomic 8067244765
Dont swallow API errors (fixes #1834) (#1837)
Dont swallow API errors (fixes #1834)
3 years ago
Dessalines 8708ad1b44 Moving settings and secrets to context. 3 years ago
Nutomic f6f169b4eb
Rewrite remaining activities (#1712)
* Limit type/method visibility in apub code

* Simplify db_queries traits by removing generics

* Simplify delete activity implementation

* Rewrite delete activities

* Implement helper functions for websocket message sending

* When receiving delete reason as empty string, change to none

* Rewrite remaining activities

* Simplify inbox

* Remove struct ActivityCommonFields, derive ActivityFields trait instead

* Community should announce received activities to followers
3 years ago
Nutomic 28d6370c35
Rewrite delete activities (#1699)
* Limit type/method visibility in apub code

* Simplify db_queries traits by removing generics

* Simplify delete activity implementation

* Rewrite delete activities

* Implement helper functions for websocket message sending

* When receiving delete reason as empty string, change to none
3 years ago
Dessalines d85f0d087e
Blank out extra info for deleted or removed content. Fixes #1679 (#1680) 3 years ago
Dessalines f54209c451
Running clippy --fix (#1647) 3 years ago
Dessalines f8cd6fd445 Making more fields optional in the API.
- Fixes #1569
3 years ago