Commit Graph

54 Commits (375d9a2a3cf9c023ac6ac832d98f1b456fb9e2ce)

Author SHA1 Message Date
phiresky 375d9a2a3c
Persistent, performant, reliable federation queue (#3605)
* persistent activity queue

* fixes

* fixes

* make federation workers function callable from outside

* log federation instances

* dead instance detection not needed here

* taplo fmt

* split federate bin/lib

* minor fix

* better logging

* log

* create struct to hold cancellable task for readability

* use boxfuture for readability

* reset submodule

* fix

* fix lint

* swap

* remove json column, use separate array columns instead

* some review comments

* make worker a struct for readability

* minor readability

* add local filter to community follower view

* remove separate lemmy_federate entry point

* fix remaining duration

* address review comments mostly

* fix lint

* upgrade actitypub-fed to simpler interface

* fix sql format

* increase delays a bit

* fixes after merge

* remove selectable

* fix instance selectable

* add comment

* start federation based on latest id at the time

* rename federate process args

* dead instances in one query

* filter follow+report activities by local

* remove synchronous federation

remove activity sender queue

* lint

* fix federation tests by waiting for results to change

* fix fed test

* fix comment report

* wait some more

* Apply suggestions from code review

Co-authored-by: SorteKanin <sortekanin@gmail.com>

* fix most remaining tests

* wait until private messages

* fix community tests

* fix community tests

* move arg parse

* use instance_id instead of domain in federation_queue_state table

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: SorteKanin <sortekanin@gmail.com>
8 months ago
Nutomic 71d61138bc
Replace ammonia lib with manual html escaping (fixes #3774) (#3938)
* Replace ammonia lib with manual html escaping (fixes #3774)

* prettier

* clippy

* remove sanitize unit test

* fix tests

* fix schema
9 months ago
Nutomic 3471f3533c
Sanitize html (#3708)
* HTML sanitization in apub code

* Sanitize API inputs

* fmt

* Dont allow html a, img tags

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
10 months ago
Nutomic afac3ee7ff
Skip fragile API tests (#3723)
* Skip fragile API tests

These tests fail very often for no reason, eg in #3712 and #3696.
Better to disable them until they can be fixed.

* killall -s1
10 months ago
Dessalines e17f0097ff
Fix federation test errors with new lemmy-js-client (#3678) 10 months ago
Nutomic 047db9ac85
Handle displaying of deleted and removed posts/comments (fixes #2624) (#3286)
* Handle displaying of deleted and removed posts/comments (fixes #2624)

* remove duplicate test

* fix tests

* no show_removed/show_deleted

* merge

* partially fix tests

* fix tests

* clippy

* fix tests

* get rid of build_post_response_deleted_allowed
10 months ago
Dessalines 2a088cf023
Update prettier to 3.0.0 (#3509)
* Update prettier to 3.0.0

* Updating prettier dep.

* Explicity 3.0.0 version.
11 months ago
Dessalines 88a0d2feec
Adding typescript generation for API. Fixes #2824 (#2827)
* Adding typescript generation for API. Fixes #2824

* Try to fix Ltree issue 1.

* Forgot a few types.

* Fixing api tests.

* Removing url_serde line.

* Manually deriving TS for some types.
1 year ago
Dessalines 48f187188b
Show deleted and removed posts for profile views. Fixes #2624 (#2729)
* Show deleted and removed posts for profile views. Fixes #2624

* Only showing non-deleted/removed posts for creator.

* Add a admin or mod check to views, to show deleted and removed posts.

- Also removed the pointless "blanking" functions

* Fix clippy

* Make hidden posts comment clearer.

* Fixing federation tests.

* Fixing fmt.
1 year ago
Dessalines 0630d214e3
Removing sniptt/monads for lemmy-js-client. (#2644)
* Removing sniptt/monads for lemmy-js-client.

* Fix tests.
1 year ago
Anon 9dfd819691
Add support for Featured Posts (#2585)
* Add support for Featured Posts

* Fix rebase

* More fixes
1 year ago
Dessalines 235cc8b228
Moving settings to Database. (#2492)
* Moving settings to Database.

- Moves many settings into the database. Fixes #2285
- Adds a local_site and instance table. Fixes #2365 . Fixes #2368
- Separates SQL update an insert forms, to avoid runtime errors.
- Adds TypedBuilder to all the SQL forms, instead of default.

* Fix weird clippy issue.

* Removing extra lines.

* Some fixes from suggestions.

* Fixing apub tests.

* Using instance creation helper function.

* Move forms to their own line.

* Trying to fix local_site_data, still broken.

* Fixing federation tests.

* Trying to fix check features 1.

* Addressing PR comments.

* Adding check_apub to all verify functions.
2 years ago
Nutomic 6c3e984ad1
Only allow authenticated users to fetch remote objects (#2493)
* Only allow authenticated users to fetch remote objects

* try to fix api tests
2 years ago
Dessalines 9c3efe32e7
First pass at adding comment trees. (#2362)
* First pass at adding comment trees.

- Extracted comment replies into its own table.
- Added ltree column to comment
- Added parent_id param to GetComments to fetch a tree branch
- No paging / limiting yet

* Adding child_count to comment_aggregates.

* Adding parent comment update counts

* Fix unit tests.

* Comment tree paging mostly done.

* Fix clippy

* Fix drone tests wrong postgres version.

* Fix unit tests.

* Add back in delete in unit test.

* Add postgres upgrade script.

* Fixing some PR comments.

* Move update ltree into Comment::create

* Updating based on comments.

* Fix send soft fail.
2 years ago
Nutomic a2a594b763
Extract Activitypub logic into separate library (#2288)
* Create example for apub lib

* some rewriting of apub lib

* Add LocalInstance struct for apub lib to avoid using Lemmy Settings

* Move ActorType trait to lemmy_apub, because its not needed in library

* Use reqwest_retry instead of custom impl, dont specify timeout on every send()

* Some improvements to example

* Moved inbox handling to library

* bug fixes

* Move context and serde helpers into library

* wip: example changes

* Add lemmy_utils feature to build only LemmyError

* Rename to activitypub_federation

* Remove lemmy_utils dep from activitypub_federation using generic error type

* Finish activitypub example

* Cleanup and fix tests

* Reorganize library files

* Remove ApubObject.to_tombstone()

* Extract activitypub library into separate git repository
2 years ago
Dessalines eafb3869f2
Revert "Add pending, and change use specific API response for FollowCommunity…" (#2283) 2 years ago
Dessalines ed9b66541e Fixing API tests. 2 years ago
Nutomic 65cac21713
Fix verify_mod_action check for remote admin actions (#2190)
* Fix verify_mod_action check for remote admin actions

* fix federation test
2 years ago
Dessalines 500ca15893 Trying to fix api tests 2 2 years ago
Dessalines 5698dd7771 Trying to fix api tests 1 2 years ago
Nutomic dd865c5af5
Implement instance actor (#1798)
* Implement instance actor

* wip: make site bans federate

* finish implementation and unit tests for federated bans

* start adding api tests

* fix api test

* remve site from GetCommunityResponse

* only federate site bans originating from user's home instance

* dont expose site.private_key in api
2 years ago
Dessalines d9ecabee87
Dont blank out post or community info. Fixes #1813 (#1841)
* Dont blank out post or community info. Fixes #1813

* Fix federation tests

* Only blank out info for non-logged in users.

* Remove pointless trait.

* Blank the community if deleted for post

* Fix hook

* Fixing again

* Fixing again

* Fixing again
3 years ago
Nutomic 149a4e0de8
Breaking apub changes (#1859)
* Remove CommentInReplyToMigration

* Remove compat for RemovePostCommentOrCommunity

* Remove PublicUrlMigration

* Change type of pm to ChatMessage from Pleroma, make pm.to array

* Use person.summary instead of person.content for pleroma compat

* Also change group.content to summary

* Rewrite apub object test json to serve as nice examples

* Also add test case for parsing pleroma private message
3 years ago
Nutomic 18badcfdb4
Federate reports (#1830)
* Federate reports

* add federation test cases for reports
3 years ago
Dessalines 2402515fcc
Dont allow posts to deleted / removed communities. Fixes #1827 (#1828)
* Dont allow posts to deleted / removed communities. Fixes #1827

* Fixing couldnt find community error.

* Adding check in createorupdate post and comment.

* make sure post wasn't deleted or removed.

* Adding a post not deleted or removed check to creatorupdatecomment.

* Using pub(crate)
3 years ago
Nutomic 527eefbe92
Rewrite fetcher (#1792)
* Use new fetcher implementation for post/comment

* rewrite person fetch to use new fetcher

* rewrite community to use new fetcher

* rename new_fetcher to dereference_object_id

* make ObjectId a newtype

* handle deletion in new fetcher

* rewrite apub object search to be generic

* move upsert() method out of ApubObject trait

* simplify ObjectId::new (and fix clippy)
3 years ago
Dessalines c23e7cc20d
Fixing ResolveObject API and unit tests (#1713) 3 years ago
Dessalines d85f0d087e
Blank out extra info for deleted or removed content. Fixes #1679 (#1680) 3 years ago
Nutomic aa79c5131f
Implement federated bans (fixes #1298) (#1553)
* Implement federated bans (fixes #1298)

* mod actions should always be federated to affected user, in addition to followers

* Make Undo/Block work for remote mods

* clippy fix

* fix federation test

* vscodium doesnt auto-save changes...
3 years ago
Felix Ableitner 4f54108a9c Merge branch 'main' into federated-moderation 3 years ago
Felix Ableitner 4f7dca7c2b Fix three federation test cases 3 years ago
Dessalines 434fb53dd1 Trying to fix API tests. 3 years ago
Dessalines d767dd998e Merge branch 'drone-io-dess' into move_views_to_diesel_drone 3 years ago
Dessalines 5af8257e19 Changing unit tests to api v2. 3 years ago
Dessalines 9d0709dfe8 Trying again. 3 years ago
Dessalines 2e5297e337 Trying again. 3 years ago
Dessalines 5c266302c5 Adding unfollows. 3 years ago
Dessalines caaf6b178b Trying again. 3 years ago
Dessalines 583808d5e7 Trying again. 3 years ago
Dessalines 5768a4eda7 Trying again. 3 years ago
Dessalines 4997d4b0b5 Trying again. 3 years ago
Dessalines 179709cc09 Fixing drone tests. 3 years ago
Dessalines ccd26bfcd7 Trying to fix post test again. 3 years ago
Dessalines f6ba6d5590 Trying to fix post test again. 3 years ago
Dessalines dfe17662df Trying to fix post test again. 3 years ago
Dessalines 28c217eb66 Trying to fix post test again. 3 years ago
Dessalines d594005d49 Trying to fix post test again. 3 years ago
Dessalines 4410d2e44d Trying to fix post test again. 3 years ago
Dessalines 5dff60adc5 Trying to fix post test again. 3 years ago
Dessalines 9e5824df85 Trying to fix post test. 3 years ago