Merge remote-tracking branch 'origin/develop' into global-status-expiration

2020-06-01 15:48:51 +04:00 · 2020-06-01 15:48:51 +04:00 · a7627bdc7a
parent decaa64f75 e96765df6b
commit a7627bdc7a
921 changed files with 18591 additions and 8370 deletions
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@ -48,6 +48,7 @@ benchmark:

 unit-testing:
  stage: test
+  retry: 2
  cache: &testing_cache_policy
    <<: *global_cache_policy
    policy: pull
@ -80,6 +81,7 @@ unit-testing:

 unit-testing-rum:
  stage: test
+  retry: 2
  cache: *testing_cache_policy
  services:
  - name: minibikini/postgres-with-rum:12
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -6,6 +6,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 ## [unreleased]

 ### Changed
+- MFR policy to set global expiration for all local Create activities
 <details>
  <summary>API Changes</summary>
 - **Breaking:** Emoji API: changed methods and renamed routes.
@ -15,14 +16,19 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - **Breaking:** removed `with_move` parameter from notifications timeline.

 ### Added
+- ActivityPub: Added support for existing AP ids for instances migrated from Mastodon.
+- Instance: Add `background_image` to configuration and `/api/v1/instance`
 - Instance: Extend `/api/v1/instance` with Pleroma-specific information.
 - NodeInfo: `pleroma:api/v1/notifications:include_types_filter` to the `features` list.
 - NodeInfo: `pleroma_emoji_reactions` to the `features` list.
 - Configuration: `:restrict_unauthenticated` setting, restrict access for unauthenticated users to timelines (public and federate), user profiles and statuses.
+- Configuration: Add `:database_config_whitelist` setting to whitelist settings which can be configured from AdminFE.
+- Configuration: `filename_display_max_length` option to set filename truncate limit, if filename display enabled (0 = no limit).
 - New HTTP adapter [gun](https://github.com/ninenines/gun). Gun adapter requires minimum OTP version of 22.2 otherwise Pleroma won’t start. For hackney OTP update is not required.
 - Mix task to create trusted OAuth App.
 - Notifications: Added `follow_request` notification type.
 - Added `:reject_deletes` group to SimplePolicy
+- MRF (`EmojiStealPolicy`): New MRF Policy which allows to automatically download emojis from remote instances
 <details>
  <summary>API Changes</summary>
 - Mastodon API: Extended `/api/v1/instance`.
@ -30,6 +36,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Mastodon API: Added `/api/v1/notifications/:id/dismiss` endpoint.
 - Mastodon API: Add support for filtering replies in public and home timelines
 - Admin API: endpoints for create/update/delete OAuth Apps.
+- Admin API: endpoint for status view.
 </details>

 ### Fixed
@ -37,27 +44,48 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - **Breaking**: SimplePolicy `:reject` and `:accept` allow deletions again
 - Fix follower/blocks import when nicknames starts with @
 - Filtering of push notifications on activities from blocked domains
+- Resolving Peertube accounts with Webfinger
+- `blob:` urls not being allowed by connect-src CSP

-### Changed
- MFR policy to set global expiration for all local Create activities
+## [Unreleased (patch)]
+
+### Fixed
+- Healthcheck reporting the number of memory currently used, rather than allocated in total
+- `InsertSkeletonsForDeletedUsers` failing on some instances
+
+## [2.0.3] - 2020-05-02

-## [unreleased-patch]
 ### Security
 - Disallow re-registration of previously deleted users, which allowed viewing direct messages addressed to them
 - Mastodon API: Fix `POST /api/v1/follow_requests/:id/authorize` allowing to force a follow from a local user even if they didn't request to follow
+- CSP: Sandbox uploads

 ### Fixed
- Logger configuration through AdminFE
+- Notifications from blocked domains
+- Potential federation issues with Mastodon versions before 3.0.0
 - HTTP Basic Authentication permissions issue
+- Follow/Block imports not being able to find the user if the nickname started with an `@`
+- Instance stats counting internal users
+- Inability to run a From Source release without git
 - ObjectAgePolicy didn't filter out old messages
+- `blob:` urls not being allowed by CSP

 ### Added
 - NodeInfo: ObjectAgePolicy settings to the `federation` list.
+- Follow request notifications
 <details>
  <summary>API Changes</summary>
 - Admin API: `GET /api/pleroma/admin/need_reboot`.
 </details>

+### Upgrade notes
+
+1. Restart Pleroma
+2. Run database migrations (inside Pleroma directory):
+  - OTP: `./bin/pleroma_ctl migrate`
+  - From Source: `mix ecto.migrate`
+
+
 ## [2.0.2] - 2020-04-08
 ### Added
 - Support for Funkwhale's `Audio` activity
@ -156,6 +184,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Mastodon API: `pleroma.thread_muted` to the Status entity
 - Mastodon API: Mark the direct conversation as read for the author when they send a new direct message
 - Mastodon API, streaming: Add `pleroma.direct_conversation_id` to the `conversation` stream event payload.
+- Mastodon API: Add `pleroma.unread_count` to the Marker entity
 - Admin API: Render whole status in grouped reports
 - Mastodon API: User timelines will now respect blocks, unless you are getting the user timeline of somebody you blocked (which would be empty otherwise).
 - Mastodon API: Favoriting / Repeating a post multiple times will now return the identical response every time. Before, executing that action twice would return an error ("already favorited") on the second try.
--- a/benchmarks/load_testing/activities.ex
+++ b/benchmarks/load_testing/activities.ex
@ -123,7 +123,7 @@ def generate_tagged_activities(opts \\ []) do
    Enum.each(1..activity_count, fn _ ->
      random = :rand.uniform()
      i = Enum.find_index(intervals, fn {lower, upper} -> lower <= random && upper > random end)
-      CommonAPI.post(Enum.random(users), %{"status" => "a post with the tag #tag_#{i}"})
+      CommonAPI.post(Enum.random(users), %{status: "a post with the tag #tag_#{i}"})
    end)
  end

@ -137,8 +137,8 @@ defp generate_long_thread(visibility, user, friends, non_friends, _opts) do

    {:ok, activity} =
      CommonAPI.post(user, %{
-        "status" => "Start of #{visibility} long thread",
-        "visibility" => visibility
+        status: "Start of #{visibility} long thread",
+        visibility: visibility
      })

    Agent.update(:benchmark_state, fn state ->
@ -186,7 +186,7 @@ defp insert_activity("simple", visibility, group, user, friends, non_friends, _o
    {:ok, _activity} =
      group
      |> get_actor(user, friends, non_friends)
-      |> CommonAPI.post(%{"status" => "Simple status", "visibility" => visibility})
+      |> CommonAPI.post(%{status: "Simple status", visibility: visibility})
  end

  defp insert_activity("emoji", visibility, group, user, friends, non_friends, _opts) do
@ -194,8 +194,8 @@ defp insert_activity("emoji", visibility, group, user, friends, non_friends, _op
      group
      |> get_actor(user, friends, non_friends)
      |> CommonAPI.post(%{
-        "status" => "Simple status with emoji :firefox:",
-        "visibility" => visibility
+        status: "Simple status with emoji :firefox:",
+        visibility: visibility
      })
  end

@ -213,8 +213,8 @@ defp insert_activity("mentions", visibility, group, user, friends, non_friends,
      group
      |> get_actor(user, friends, non_friends)
      |> CommonAPI.post(%{
-        "status" => Enum.join(user_mentions, ", ") <> " simple status with mentions",
-        "visibility" => visibility
+        status: Enum.join(user_mentions, ", ") <> " simple status with mentions",
+        visibility: visibility
      })
  end

@ -236,8 +236,8 @@ defp insert_activity("hell_thread", visibility, group, user, friends, non_friend
      group
      |> get_actor(user, friends, non_friends)
      |> CommonAPI.post(%{
-        "status" => mentions <> " hell thread status",
-        "visibility" => visibility
+        status: mentions <> " hell thread status",
+        visibility: visibility
      })
  end

@ -262,9 +262,9 @@ defp insert_activity("attachment", visibility, group, user, friends, non_friends

    {:ok, _activity} =
      CommonAPI.post(actor, %{
-        "status" => "Post with attachment",
-        "visibility" => visibility,
-        "media_ids" => [object.id]
+        status: "Post with attachment",
+        visibility: visibility,
+        media_ids: [object.id]
      })
  end

@ -272,7 +272,7 @@ defp insert_activity("tag", visibility, group, user, friends, non_friends, _opts
    {:ok, _activity} =
      group
      |> get_actor(user, friends, non_friends)
-      |> CommonAPI.post(%{"status" => "Status with #tag", "visibility" => visibility})
+      |> CommonAPI.post(%{status: "Status with #tag", visibility: visibility})
  end

  defp insert_activity("like", visibility, group, user, friends, non_friends, opts) do
@ -312,8 +312,7 @@ defp insert_activity("simple_thread", visibility, group, user, friends, non_frie
    actor = get_actor(group, user, friends, non_friends)
    tasks = get_reply_tasks(visibility, group)

-    {:ok, activity} =
-      CommonAPI.post(user, %{"status" => "Simple status", "visibility" => visibility})
+    {:ok, activity} = CommonAPI.post(user, %{status: "Simple status", visibility: visibility})

    acc = {activity.id, ["@" <> actor.nickname, "reply to status"]}
    insert_replies(tasks, visibility, user, friends, non_friends, acc)
@ -336,8 +335,8 @@ defp insert_activity("simple_thread", "direct", group, user, friends, non_friend

    {:ok, activity} =
      CommonAPI.post(actor, %{
-        "status" => Enum.join(data, ", ") <> "simple status",
-        "visibility" => "direct"
+        status: Enum.join(data, ", ") <> "simple status",
+        visibility: "direct"
      })

    acc = {activity.id, ["@" <> user.nickname | data] ++ ["reply to status"]}
@ -527,9 +526,9 @@ defp insert_direct_replies(tasks, user, list, acc) do
  defp insert_reply(actor, data, activity_id, visibility) do
    {:ok, reply} =
      CommonAPI.post(actor, %{
-        "status" => Enum.join(data, ", "),
-        "visibility" => visibility,
-        "in_reply_to_status_id" => activity_id
+        status: Enum.join(data, ", "),
+        visibility: visibility,
+        in_reply_to_status_id: activity_id
      })

    {reply.id, ["@" <> actor.nickname | data]}
--- a/benchmarks/load_testing/fetcher.ex
+++ b/benchmarks/load_testing/fetcher.ex
@ -387,56 +387,47 @@ defp render_timelines(user) do

    favourites = ActivityPub.fetch_favourites(user)

-    output_relationships =
-      !!Pleroma.Config.get([:extensions, :output_relationships_in_statuses_by_default])
-
    Benchee.run(
      %{
        "Rendering home timeline" => fn ->
          StatusView.render("index.json", %{
            activities: home_activities,
            for: user,
-            as: :activity,
-            skip_relationships: !output_relationships
+            as: :activity
          })
        end,
        "Rendering direct timeline" => fn ->
          StatusView.render("index.json", %{
            activities: direct_activities,
            for: user,
-            as: :activity,
-            skip_relationships: !output_relationships
+            as: :activity
          })
        end,
        "Rendering public timeline" => fn ->
          StatusView.render("index.json", %{
            activities: public_activities,
            for: user,
-            as: :activity,
-            skip_relationships: !output_relationships
+            as: :activity
          })
        end,
        "Rendering tag timeline" => fn ->
          StatusView.render("index.json", %{
            activities: tag_activities,
            for: user,
-            as: :activity,
-            skip_relationships: !output_relationships
+            as: :activity
          })
        end,
        "Rendering notifications" => fn ->
          Pleroma.Web.MastodonAPI.NotificationView.render("index.json", %{
            notifications: notifications,
-            for: user,
-            skip_relationships: !output_relationships
+            for: user
          })
        end,
        "Rendering favourites timeline" => fn ->
          StatusView.render("index.json", %{
            activities: favourites,
            for: user,
-            as: :activity,
-            skip_relationships: !output_relationships
+            as: :activity
          })
        end
      },
--- a/benchmarks/load_testing/users.ex
+++ b/benchmarks/load_testing/users.ex
@ -55,7 +55,7 @@ defp generate_user(i) do
      name: "Test テスト User #{i}",
      email: "user#{i}@example.com",
      nickname: "nick#{i}",
-      password_hash: Comeonin.Pbkdf2.hashpwsalt("test"),
+      password_hash: Pbkdf2.hash_pwd_salt("test"),
      bio: "Tester Number #{i}",
      local: !remote
    }
--- a/config/config.exs
+++ b/config/config.exs
@ -71,7 +71,8 @@
      follow_redirect: true,
      pool: :upload
    ]
-  ]
+  ],
+  filename_display_max_length: 30

 config :pleroma, Pleroma.Uploaders.Local, uploads: "uploads"

@ -183,6 +184,7 @@
  email: "example@example.com",
  notify_email: "noreply@example.com",
  description: "A Pleroma instance, an alternative fediverse server",
+  background_image: "/images/city.jpg",
  limit: 5_000,
  chat_limit: 5_000,
  remote_limit: 100_000,
@ -238,9 +240,18 @@
  account_field_value_length: 2048,
  external_user_synchronization: true,
  extended_nickname_format: true,
-  cleanup_attachments: false
-
-config :pleroma, :extensions, output_relationships_in_statuses_by_default: true
+  cleanup_attachments: false,
+  multi_factor_authentication: [
+    totp: [
+      # digits 6 or 8
+      digits: 6,
+      period: 30
+    ],
+    backup_codes: [
+      number: 5,
+      length: 16
+    ]
+  ]

 config :pleroma, :feed,
  post_title: %{
@ -262,20 +273,33 @@

 config :pleroma, :frontend_configurations,
  pleroma_fe: %{
-    theme: "pleroma-dark",
-    logo: "/static/logo.png",
+    alwaysShowSubjectInput: true,
    background: "/images/city.jpg",
-    redirectRootNoLogin: "/main/all",
-    redirectRootLogin: "/main/friends",
-    showInstanceSpecificPanel: true,
-    scopeOptionsEnabled: false,
-    formattingOptionsEnabled: false,
    collapseMessageWithSubject: false,
+    disableChat: false,
+    greentext: false,
+    hideFilteredStatuses: false,
+    hideMutedPosts: false,
    hidePostStats: false,
+    hideSitename: false,
    hideUserStats: false,
+    loginMethod: "password",
+    logo: "/static/logo.png",
+    logoMargin: ".1em",
+    logoMask: true,
+    minimalScopesMode: false,
+    noAttachmentLinks: false,
+    nsfwCensorImage: "",
+    postContentType: "text/plain",
+    redirectRootLogin: "/main/friends",
+    redirectRootNoLogin: "/main/all",
    scopeCopy: true,
+    sidebarRight: false,
+    showFeaturesPanel: true,
+    showInstanceSpecificPanel: false,
    subjectLineBehavior: "email",
-    alwaysShowSubjectInput: true
+    theme: "pleroma-dark",
+    webPushNotifications: false
  },
  masto_fe: %{
    showInstanceSpecificPanel: true
@ -369,6 +393,10 @@

 config :pleroma, :media_proxy,
  enabled: false,
+  invalidation: [
+    enabled: false,
+    provider: Pleroma.Web.MediaProxy.Invalidation.Script
+  ],
  proxy_opts: [
    redirect_on_failure: false,
    max_body_length: 25 * 1_048_576,
@ -655,6 +683,8 @@
  profiles: %{local: false, remote: false},
  activities: %{local: false, remote: false}

+config :pleroma, Pleroma.Web.ApiSpec.CastAndValidate, strict: false
+
 # Import environment specific config. This must remain at the bottom
 # of this file so it overrides the configuration defined above.
 import_config "#{Mix.env()}.exs"
--- a/config/description.exs
+++ b/config/description.exs
@ -28,7 +28,8 @@
      %{
        key: :filters,
        type: {:list, :module},
-        description: "List of filter modules for uploads",
+        description:
+          "List of filter modules for uploads. Module names are shortened (removed leading `Pleroma.Upload.Filter.` part), but on adding custom module you need to use full name.",
        suggestions:
          Generator.list_modules_in_dir(
            "lib/pleroma/upload/filter",
@ -118,6 +119,11 @@
            ]
          }
        ]
+      },
+      %{
+        key: :filename_display_max_length,
+        type: :integer,
+        description: "Set max length of a filename to display. 0 = no limit. Default: 30"
      }
    ]
  },
@ -678,14 +684,6 @@
          7
        ]
      },
-      %{
-        key: :federation_publisher_modules,
-        type: {:list, :module},
-        description: "List of modules for federation publishing",
-        suggestions: [
-          Pleroma.Web.ActivityPub.Publisher
-        ]
-      },
      %{
        key: :allow_relay,
        type: :boolean,
@ -694,7 +692,8 @@
      %{
        key: :rewrite_policy,
        type: [:module, {:list, :module}],
-        description: "A list of MRF policies enabled",
+        description:
+          "A list of enabled MRF policies. Module names are shortened (removed leading `Pleroma.Web.ActivityPub.MRF.` part), but on adding custom module you need to use full name.",
        suggestions:
          Generator.list_modules_in_dir(
            "lib/pleroma/web/activity_pub/mrf",
@ -712,7 +711,7 @@
        key: :quarantined_instances,
        type: {:list, :string},
        description:
-          "List of ActivityPub instances where private (DMs, followers-only) activities will not be send",
+          "List of ActivityPub instances where private (DMs, followers-only) activities will not be sent",
        suggestions: [
          "quarantined.com",
          "*.quarantined.com"
@ -919,6 +918,69 @@
        key: :external_user_synchronization,
        type: :boolean,
        description: "Enabling following/followers counters synchronization for external users"
+      },
+      %{
+        key: :multi_factor_authentication,
+        type: :keyword,
+        description: "Multi-factor authentication settings",
+        suggestions: [
+          [
+            totp: [digits: 6, period: 30],
+            backup_codes: [number: 5, length: 16]
+          ]
+        ],
+        children: [
+          %{
+            key: :totp,
+            type: :keyword,
+            description: "TOTP settings",
+            suggestions: [digits: 6, period: 30],
+            children: [
+              %{
+                key: :digits,
+                type: :integer,
+                suggestions: [6],
+                description:
+                  "Determines the length of a one-time pass-code, in characters. Defaults to 6 characters."
+              },
+              %{
+                key: :period,
+                type: :integer,
+                suggestions: [30],
+                description:
+                  "a period for which the TOTP code will be valid, in seconds. Defaults to 30 seconds."
+              }
+            ]
+          },
+          %{
+            key: :backup_codes,
+            type: :keyword,
+            description: "MFA backup codes settings",
+            suggestions: [number: 5, length: 16],
+            children: [
+              %{
+                key: :number,
+                type: :integer,
+                suggestions: [5],
+                description: "number of backup codes to generate."
+              },
+              %{
+                key: :length,
+                type: :integer,
+                suggestions: [16],
+                description:
+                  "Determines the length of backup one-time pass-codes, in characters. Defaults to 16 characters."
+              }
+            ]
+          }
+        ]
+      },
+      %{
+        key: :instance_thumbnail,
+        type: :string,
+        description:
+          "The instance thumbnail image. It will appear in [Pleroma Instances](http://distsn.org/pleroma-instances.html)",
+        suggestions: ["/instance/thumbnail.jpeg"]
      }
    ]
  },
@ -1046,32 +1108,98 @@
        description: "Settings for Pleroma FE",
        suggestions: [
          %{
-            theme: "pleroma-dark",
-            logo: "/static/logo.png",
-            background: "/images/city.jpg",
-            redirectRootNoLogin: "/main/all",
-            redirectRootLogin: "/main/friends",
-            showInstanceSpecificPanel: true,
-            scopeOptionsEnabled: false,
-            formattingOptionsEnabled: false,
-            collapseMessageWithSubject: false,
-            hidePostStats: false,
-            hideUserStats: false,
-            scopeCopy: true,
-            subjectLineBehavior: "email",
            alwaysShowSubjectInput: true,
-            logoMask: false,
+            background: "/static/aurora_borealis.jpg",
+            collapseMessageWithSubject: false,
+            disableChat: false,
+            greentext: false,
+            hideFilteredStatuses: false,
+            hideMutedPosts: false,
+            hidePostStats: false,
+            hideSitename: false,
+            hideUserStats: false,
+            loginMethod: "password",
+            logo: "/static/logo.png",
            logoMargin: ".1em",
-            stickers: false,
-            enableEmojiPicker: false
+            logoMask: true,
+            minimalScopesMode: false,
+            noAttachmentLinks: false,
+            nsfwCensorImage: "/static/img/nsfw.74818f9.png",
+            postContentType: "text/plain",
+            redirectRootLogin: "/main/friends",
+            redirectRootNoLogin: "/main/all",
+            scopeCopy: true,
+            sidebarRight: false,
+            showFeaturesPanel: true,
+            showInstanceSpecificPanel: false,
+            subjectLineBehavior: "email",
+            theme: "pleroma-dark",
+            webPushNotifications: false
          }
        ],
        children: [
          %{
-            key: :theme,
+            key: :alwaysShowSubjectInput,
+            label: "Always show subject input",
+            type: :boolean,
+            description: "When disabled, auto-hide the subject field if it's empty"
+          },
+          %{
+            key: :background,
            type: :string,
-            description: "Which theme to use, they are defined in styles.json",
-            suggestions: ["pleroma-dark"]
+            description:
+              "URL of the background, unless viewing a user profile with a background that is set",
+            suggestions: ["/images/city.jpg"]
+          },
+          %{
+            key: :collapseMessageWithSubject,
+            label: "Collapse message with subject",
+            type: :boolean,
+            description:
+              "When a message has a subject (aka Content Warning), collapse it by default"
+          },
+          %{
+            key: :disableChat,
+            label: "PleromaFE Chat",
+            type: :boolean,
+            description: "Disables PleromaFE Chat component"
+          },
+          %{
+            key: :greentext,
+            label: "Greentext",
+            type: :boolean,
+            description: "Enables green text on lines prefixed with the > character."
+          },
+          %{
+            key: :hideFilteredStatuses,
+            label: "Hide Filtered Statuses",
+            type: :boolean,
+            description: "Hides filtered statuses from timelines."
+          },
+          %{
+            key: :hideMutedPosts,
+            label: "Hide Muted Posts",
+            type: :boolean,
+            description: "Hides muted statuses from timelines."
+          },
+          %{
+            key: :hidePostStats,
+            label: "Hide post stats",
+            type: :boolean,
+            description: "Hide notices statistics (repeats, favorites, ...)"
+          },
+          %{
+            key: :hideSitename,
+            label: "Hide Sitename",
+            type: :boolean,
+            description: "Hides instance name from PleromaFE banner."
+          },
+          %{
+            key: :hideUserStats,
+            label: "Hide user stats",
+            type: :boolean,
+            description:
+              "Hide profile statistics (posts, posts per day, followers, followings, ...)"
          },
          %{
            key: :logo,
@ -1080,11 +1208,44 @@
            suggestions: ["/static/logo.png"]
          },
          %{
-            key: :background,
+            key: :logoMargin,
+            label: "Logo margin",
            type: :string,
            description:
-              "URL of the background, unless viewing a user profile with a background that is set",
-            suggestions: ["/images/city.jpg"]
+              "Allows you to adjust vertical margins between logo boundary and navbar borders. " <>
+                "The idea is that to have logo's image without any extra margins and instead adjust them to your need in layout.",
+            suggestions: [".1em"]
+          },
+          %{
+            key: :logoMask,
+            label: "Logo mask",
+            type: :boolean,
+            description:
+              "By default it assumes logo used will be monochrome with alpha channel to be compatible with both light and dark themes. " <>
+                "If you want a colorful logo you must disable logoMask."
+          },
+          %{
+            key: :minimalScopesMode,
+            label: "Minimal scopes mode",
+            type: :boolean,
+            description:
+              "Limit scope selection to Direct, User default, and Scope of post replying to. " <>
+                "Also prevents replying to a DM with a public post from PleromaFE."
+          },
+          %{
+            key: :nsfwCensorImage,
+            label: "NSFW Censor Image",
+            type: :string,
+            description:
+              "URL of the image to use for hiding NSFW media attachments in the timeline.",
+            suggestions: ["/static/img/nsfw.74818f9.png"]
+          },
+          %{
+            key: :postContentType,
+            label: "Post Content Type",
+            type: {:dropdown, :atom},
+            description: "Default post formatting option.",
+            suggestions: ["text/plain", "text/html", "text/markdown", "text/bbcode"]
          },
          %{
            key: :redirectRootNoLogin,
@ -1102,51 +1263,31 @@
              "Relative URL which indicates where to redirect when a user is logged in",
            suggestions: ["/main/friends"]
          },
-          %{
-            key: :showInstanceSpecificPanel,
-            label: "Show instance specific panel",
-            type: :boolean,
-            description: "Whenether to show the instance's specific panel"
-          },
-          %{
-            key: :scopeOptionsEnabled,
-            label: "Scope options enabled",
-            type: :boolean,
-            description: "Enable setting a notice visibility and subject/CW when posting"
-          },
-          %{
-            key: :formattingOptionsEnabled,
-            label: "Formatting options enabled",
-            type: :boolean,
-            description:
-              "Enable setting a formatting different than plain-text (ie. HTML, Markdown) when posting, relates to `:instance`, `allowed_post_formats`"
-          },
-          %{
-            key: :collapseMessageWithSubject,
-            label: "Collapse message with subject",
-            type: :boolean,
-            description:
-              "When a message has a subject (aka Content Warning), collapse it by default"
-          },
-          %{
-            key: :hidePostStats,
-            label: "Hide post stats",
-            type: :boolean,
-            description: "Hide notices statistics (repeats, favorites, ...)"
-          },
-          %{
-            key: :hideUserStats,
-            label: "Hide user stats",
-            type: :boolean,
-            description:
-              "Hide profile statistics (posts, posts per day, followers, followings, ...)"
-          },
          %{
            key: :scopeCopy,
            label: "Scope copy",
            type: :boolean,
            description: "Copy the scope (private/unlisted/public) in replies to posts by default"
          },
+          %{
+            key: :sidebarRight,
+            label: "Sidebar on Right",
+            type: :boolean,
+            description: "Change alignment of sidebar and panels to the right."
+          },
+          %{
+            key: :showFeaturesPanel,
+            label: "Show instance features panel",
+            type: :boolean,
+            description:
+              "Enables panel displaying functionality of the instance on the About page."
+          },
+          %{
+            key: :showInstanceSpecificPanel,
+            label: "Show instance specific panel",
+            type: :boolean,
+            description: "Whether to show the instance's custom panel"
+          },
          %{
            key: :subjectLineBehavior,
            label: "Subject line behavior",
@ -1158,38 +1299,10 @@
            suggestions: ["email", "masto", "noop"]
          },
          %{
-            key: :alwaysShowSubjectInput,
-            label: "Always show subject input",
-            type: :boolean,
-            description: "When disabled, auto-hide the subject field if it's empty"
-          },
-          %{
-            key: :logoMask,
-            label: "Logo mask",
-            type: :boolean,
-            description:
-              "By default it assumes logo used will be monochrome with alpha channel to be compatible with both light and dark themes. " <>
-                "If you want a colorful logo you must disable logoMask."
-          },
-          %{
-            key: :logoMargin,
-            label: "Logo margin",
+            key: :theme,
            type: :string,
-            description:
-              "Allows you to adjust vertical margins between logo boundary and navbar borders. " <>
-                "The idea is that to have logo's image without any extra margins and instead adjust them to your need in layout.",
-            suggestions: [".1em"]
-          },
-          %{
-            key: :stickers,
-            type: :boolean,
-            description: "Enables stickers."
-          },
-          %{
-            key: :enableEmojiPicker,
-            label: "Emoji picker",
-            type: :boolean,
-            description: "Enables emoji picker."
+            description: "Which theme to use. Available themes are defined in styles.json",
+            suggestions: ["pleroma-dark"]
          }
        ]
      },
@ -1245,6 +1358,12 @@
        suggestions: [
          :pleroma_fox_tan
        ]
+      },
+      %{
+        key: :default_user_avatar,
+        type: :string,
+        description: "URL of the default user avatar.",
+        suggestions: ["/images/avi.png"]
      }
    ]
  },
@ -1814,12 +1933,6 @@
      (see https://github.com/sorentwo/oban/issues/52).
    """,
    children: [
-      %{
-        key: :repo,
-        type: :module,
-        description: "Application's Ecto repo",
-        suggestions: [Pleroma.Repo]
-      },
      %{
        key: :verbose,
        type: {:dropdown, :atom},
@ -1990,7 +2103,8 @@
      %{
        key: :parsers,
        type: {:list, :module},
-        description: "List of Rich Media parsers.",
+        description:
+          "List of Rich Media parsers. Module names are shortened (removed leading `Pleroma.Web.RichMedia.Parsers.` part), but on adding custom module you need to use full name.",
        suggestions: [
          Pleroma.Web.RichMedia.Parsers.MetaTagsParser,
          Pleroma.Web.RichMedia.Parsers.OEmbed,
@ -2002,7 +2116,8 @@
        key: :ttl_setters,
        label: "TTL setters",
        type: {:list, :module},
-        description: "List of rich media TTL setters.",
+        description:
+          "List of rich media TTL setters. Module names are shortened (removed leading `Pleroma.Web.RichMedia.Parser.` part), but on adding custom module you need to use full name.",
        suggestions: [
          Pleroma.Web.RichMedia.Parser.TTL.AwsSignedUrl
        ]
@ -2592,18 +2707,6 @@
      }
    ]
  },
-  %{
-    group: :http_signatures,
-    type: :group,
-    description: "HTTP Signatures settings",
-    children: [
-      %{
-        key: :adapter,
-        type: :module,
-        suggestions: [Pleroma.Signature]
-      }
-    ]
-  },
  %{
    group: :pleroma,
    key: :http,
@ -2676,6 +2779,8 @@
      %{
        key: :scrub_policy,
        type: {:list, :module},
+        description:
+          "Module names are shortened (removed leading `Pleroma.HTML.` part), but on adding custom module you need to use full name.",
        suggestions: [Pleroma.HTML.Transform.MediaProxy, Pleroma.HTML.Scrubber.Default]
      }
    ]
@ -3210,5 +3315,19 @@
        ]
      }
    ]
+  },
+  %{
+    group: :pleroma,
+    key: Pleroma.Web.ApiSpec.CastAndValidate,
+    type: :group,
+    children: [
+      %{
+        key: :strict,
+        type: :boolean,
+        description:
+          "Enables strict input validation (useful in development, not recommended in production)",
+        suggestions: [false]
+      }
+    ]
  }
 ]
--- a/config/dev.exs
+++ b/config/dev.exs
@ -52,6 +52,8 @@
  hostname: "localhost",
  pool_size: 10

+config :pleroma, Pleroma.Web.ApiSpec.CastAndValidate, strict: true
+
 if File.exists?("./config/dev.secret.exs") do
  import_config "dev.secret.exs"
 else
--- a/config/test.exs
+++ b/config/test.exs
@ -56,6 +56,19 @@
  ignore_hosts: [],
  ignore_tld: ["local", "localdomain", "lan"]

+config :pleroma, :instance,
+  multi_factor_authentication: [
+    totp: [
+      # digits 6 or 8
+      digits: 6,
+      period: 30
+    ],
+    backup_codes: [
+      number: 2,
+      length: 6
+    ]
+  ]
+
 config :web_push_encryption, :vapid_details,
  subject: "mailto:administrator@example.com",
  public_key:
@ -96,6 +109,8 @@

 config :pleroma, Pleroma.Plugs.RemoteIp, enabled: false

+config :pleroma, Pleroma.Web.ApiSpec.CastAndValidate, strict: true
+
 if File.exists?("./config/test.secret.exs") do
  import_config "test.secret.exs"
 else
--- a/docs/API/admin_api.md
+++ b/docs/API/admin_api.md
@ -409,6 +409,7 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret

 ### Get a password reset token for a given nickname

+
 - Params: none
 - Response:

@ -427,6 +428,14 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
  - `nicknames`
 - Response: none (code `204`)

+## PUT `/api/pleroma/admin/users/disable_mfa`
+
+### Disable mfa for user's account.
+
+- Params:
+  - `nickname`
+- Response: User’s nickname
+
 ## `GET /api/pleroma/admin/users/:nickname/credentials`

 ### Get the user's email, password, display and settings-related fields
@ -502,7 +511,23 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
  - `discoverable`
  - `actor_type`

- Response: none (code `200`)
+- Response:
+
+```json
+{"status": "success"}
+```
+
+```json
+{"errors":
+  {"actor_type": "is invalid"},
+  {"email": "has invalid format"},
+  ...
+ }
+```
+
+```json
+{"error": "Unable to update user."}
+```

 ## `GET /api/pleroma/admin/reports`

@ -755,6 +780,17 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
    - 400 Bad Request `"Invalid parameters"` when `status` is missing
  - On success: `204`, empty response

+## `GET /api/pleroma/admin/statuses/:id`
+
+### Show status by id
+
+- Params:
+  - `id`: required, status id
+- Response:
+  - On failure:
+    - 404 Not Found `"Not Found"`
+  - On success: JSON, Mastodon Status entity
+
 ## `PUT /api/pleroma/admin/statuses/:id`

 ### Change the scope of an individual reported status
--- a/docs/API/differences_in_mastoapi_responses.md
+++ b/docs/API/differences_in_mastoapi_responses.md
@ -6,10 +6,6 @@ A Pleroma instance can be identified by "<Mastodon version> (compatible; Pleroma

 Pleroma uses 128-bit ids as opposed to Mastodon's 64 bits. However just like Mastodon's ids they are lexically sortable strings

-## Attachment cap
-
-Some apps operate under the assumption that no more than 4 attachments can be returned or uploaded. Pleroma however does not enforce any limits on attachment count neither when returning the status object nor when posting.
-
 ## Timelines

 Adding the parameter `with_muted=true` to the timeline queries will also return activities by muted (not by blocked!) users.
@ -32,12 +28,20 @@ Has these additional fields under the `pleroma` object:
 - `thread_muted`: true if the thread the post belongs to is muted
 - `emoji_reactions`: A list with emoji / reaction maps. The format is `{name: "☕", count: 1, me: true}`. Contains no information about the reacting users, for that use the `/statuses/:id/reactions` endpoint.

-## Attachments
+## Media Attachments

 Has these additional fields under the `pleroma` object:

 - `mime_type`: mime type of the attachment.

+### Attachment cap
+
+Some apps operate under the assumption that no more than 4 attachments can be returned or uploaded. Pleroma however does not enforce any limits on attachment count neither when returning the status object nor when posting.
+
+### Limitations
+
+Pleroma does not process remote images and therefore cannot include fields such as `meta` and `blurhash`. It does not support focal points or aspect ratios. The frontend is expected to handle it.
+
 ## Accounts

 The `id` parameter can also be the `nickname` of the user. This only works in these endpoints, not the deeper nested ones for following etc.
@ -61,6 +65,7 @@ Has these additional fields under the `pleroma` object:
 - `deactivated`: boolean, true when the user is deactivated
 - `allow_following_move`: boolean, true when the user allows automatically follow moved following accounts
 - `unread_conversation_count`: The count of unread conversations. Only returned to the account owner.
+- `unread_notifications_count`: The count of unread notifications. Only returned to the account owner.

 ### Source

@ -215,6 +220,13 @@ Has theses additional parameters (which are the same as in Pleroma-API):
 - `avatar_upload_limit`: The same for avatars
 - `background_upload_limit`: The same for backgrounds
 - `banner_upload_limit`: The same for banners
+- `background_image`: A background image that frontends can use
 - `pleroma.metadata.features`: A list of supported features
 - `pleroma.metadata.federation`: The federation restrictions of this instance
 - `vapid_public_key`: The public key needed for push messages
+
+## Markers
+
+Has these additional fields under the `pleroma` object:
+
+- `unread_count`: contains number unread notifications
--- a/docs/API/pleroma_api.md
+++ b/docs/API/pleroma_api.md
@ -70,7 +70,49 @@ Request parameters can be passed via [query strings](https://en.wikipedia.org/wi
 * Response: JSON. Returns `{"status": "success"}` if the account was successfully disabled, `{"error": "[error message]"}` otherwise
 * Example response: `{"error": "Invalid password."}`

-## `/api/pleroma/admin/`…
+## `/api/pleroma/accounts/mfa`
+#### Gets current MFA settings
+* method: `GET`
+* Authentication: required
+* OAuth scope: `read:security`
+* Response: JSON. Returns `{"enabled": "false", "totp": false }`
+
+## `/api/pleroma/accounts/mfa/setup/totp`
+#### Pre-setup the MFA/TOTP method
+* method: `GET`
+* Authentication: required
+* OAuth scope: `write:security`
+* Response: JSON. Returns `{"key": [secret_key], "provisioning_uri": "[qr code uri]"  }` when successful, otherwise returns HTTP 422 `{"error": "error_msg"}`
+
+## `/api/pleroma/accounts/mfa/confirm/totp`
+#### Confirms & enables MFA/TOTP support for user account.
+* method: `POST`
+* Authentication: required
+* OAuth scope: `write:security`
+* Params:
+    * `password`: user's password
+    * `code`: token from TOTP App
+* Response: JSON. Returns `{}` if the enable was successful, HTTP 422 `{"error": "[error message]"}` otherwise
+
+
+## `/api/pleroma/accounts/mfa/totp`
+####  Disables MFA/TOTP method for user account.
+* method: `DELETE`
+* Authentication: required
+* OAuth scope: `write:security`
+* Params:
+    * `password`: user's password
+* Response: JSON. Returns `{}` if the disable was successful, HTTP 422 `{"error": "[error message]"}` otherwise
+* Example response: `{"error": "Invalid password."}`
+
+## `/api/pleroma/accounts/mfa/backup_codes`
+####  Generstes backup codes MFA for user account.
+* method: `GET`
+* Authentication: required
+* OAuth scope: `write:security`
+* Response: JSON. Returns `{"codes": codes}`when successful, otherwise HTTP 422 `{"error": "[error message]"}`
+
+## `/api/pleroma/admin/`
 See [Admin-API](admin_api.md)

 ## `/api/v1/pleroma/notifications/read`
@ -223,7 +265,7 @@ See [Admin-API](admin_api.md)
 * Method `PUT`
 * Authentication: required
 * Params:
-    * `image`: Multipart image
+    * `file`: Multipart image
 * Response: JSON. Returns a mastodon media attachment entity
  when successful, otherwise returns HTTP 415 `{"error": "error_msg"}`
 * Example response:
@ -316,7 +358,7 @@ The status posting endpoint takes an additional parameter, `in_reply_to_conversa
    * `recipients`: A list of ids of users that should receive posts to this conversation. This will replace the current list of recipients, so submit the full list. The owner of owner of the conversation will always be part of the set of recipients, though.
 * Response: JSON, statuses (200 - healthy, 503 unhealthy)

-## `GET /api/v1/pleroma/conversations/read`
+## `POST /api/v1/pleroma/conversations/read`
 ### Marks all user's conversations as read.
 * Method `POST`
 * Authentication: required
@ -384,7 +426,7 @@ The status posting endpoint takes an additional parameter, `in_reply_to_conversa
 * Authentication: required
 * Params:
  * `file`: file needs to be uploaded with the multipart request or link to remote file.
-  * `shortcode`: (*optional*) shortcode for new emoji, must be uniq for all emoji. If not sended, shortcode will be taken from original filename.
+  * `shortcode`: (*optional*) shortcode for new emoji, must be unique for all emoji. If not sended, shortcode will be taken from original filename.
  * `filename`: (*optional*) new emoji file name. If not specified will be taken from original filename.
 * Response: JSON, list of files for updated pack (hashmap -> shortcode => filename) with status 200, either error status with error message.

@ -494,7 +536,7 @@ Emoji reactions work a lot like favourites do. They make it possible to react to
 ```

 ## `GET /api/v1/pleroma/statuses/:id/reactions/:emoji`
-### Get an object of emoji to account mappings with accounts that reacted to the post for a specific emoji`
+### Get an object of emoji to account mappings with accounts that reacted to the post for a specific emoji
 * Method: `GET`
 * Authentication: optional
 * Params: None
--- a/docs/administration/CLI_tasks/database.md
+++ b/docs/administration/CLI_tasks/database.md
@ -69,3 +69,32 @@ mix pleroma.database update_users_following_followers_counts
 ```sh tab="From Source"
 mix pleroma.database fix_likes_collections
 ```
+
+## Vacuum the database
+
+### Analyze
+
+Running an `analyze` vacuum job can improve performance by updating statistics used by the query planner. **It is safe to cancel this.**
+
+```sh tab="OTP"
+./bin/pleroma_ctl database vacuum analyze
+```
+
+```sh tab="From Source"
+mix pleroma.database vacuum analyze
+```
+
+### Full
+
+Running a `full` vacuum job rebuilds your entire database by reading all of the data and rewriting it into smaller
+and more compact files with an optimized layout. This process will take a long time and use additional disk space as
+it builds the files side-by-side the existing database files. It can make your database faster and use less disk space,
+but should only be run if necessary. **It is safe to cancel this.**
+
+```sh tab="OTP"
+./bin/pleroma_ctl database vacuum full
+```
+
+```sh tab="From Source"
+mix pleroma.database vacuum full
+```
--- a/docs/administration/CLI_tasks/user.md
+++ b/docs/administration/CLI_tasks/user.md
@ -105,23 +105,23 @@ mix pleroma.user toggle_activated <nickname>
 ```


-## Unsubscribe local users from a user and deactivate the user
+## Deactivate a user and unsubscribes local users from the user
 ```sh tab="OTP"
- ./bin/pleroma_ctl user unsubscribe NICKNAME
+ ./bin/pleroma_ctl user deactivate NICKNAME
 ```

 ```sh tab="From Source"
-mix pleroma.user unsubscribe NICKNAME
+mix pleroma.user deactivate NICKNAME
 ```


-## Unsubscribe local users from an instance and deactivate all accounts on it
+## Deactivate all accounts from an instance and unsubscribe local users on it
 ```sh tab="OTP"
- ./bin/pleroma_ctl user unsubscribe_all_from_instance <instance>
+ ./bin/pleroma_ctl user deactivate_all_from_instance <instance>
 ```

 ```sh tab="From Source"
-mix pleroma.user unsubscribe_all_from_instance <instance>
+mix pleroma.user deactivate_all_from_instance <instance>
 ```


@ -177,4 +177,3 @@ mix pleroma.user untag <nickname> <tags>
 ```sh tab="From Source"
 mix pleroma.user toggle_confirmed <nickname>
 ```
-
--- a/docs/configuration/cheatsheet.md
+++ b/docs/configuration/cheatsheet.md
@ -8,6 +8,10 @@ For from source installations Pleroma configuration works by first importing the

 To add configuration to your config file, you can copy it from the base config. The latest version of it can be viewed [here](https://git.pleroma.social/pleroma/pleroma/blob/develop/config/config.exs). You can also use this file if you don't know how an option is supposed to be formatted.

+## :chat
+
+* `enabled` - Enables the backend chat. Defaults to `true`.
+
 ## :instance
 * `name`: The instance’s name.
 * `email`: Email used to reach an Administrator/Moderator of the instance.
@ -146,6 +150,11 @@ config :pleroma, :mrf_user_allowlist,
  * `:strip_followers` removes followers from the ActivityPub recipient list, ensuring they won't be delivered to home timelines
  * `:reject` rejects the message entirely

+#### mrf_steal_emoji
+* `hosts`: List of hosts to steal emojis from
+* `rejected_shortcodes`: Regex-list of shortcodes to reject
+* `size_limit`: File size limit (in bytes), checked before an emoji is saved to the disk
+
 #### :mrf_activity_expiration

 * `days`: Default global expiration time for all local Create activities (in days)
@ -250,6 +259,40 @@ This section describe PWA manifest instance-specific values. Currently this opti
 * `base_url`: The base URL to access a user-uploaded file. Useful when you want to proxy the media files via another host/CDN fronts.
 * `proxy_opts`: All options defined in `Pleroma.ReverseProxy` documentation, defaults to `[max_body_length: (25*1_048_576)]`.
 * `whitelist`: List of domains to bypass the mediaproxy
+* `invalidation`: options for remove media from cache after delete object:
+    * `enabled`: Enables purge cache
+    * `provider`: Which one of  the [purge cache strategy](#purge-cache-strategy) to use.
+
+### Purge cache strategy
+
+#### Pleroma.Web.MediaProxy.Invalidation.Script
+
+This strategy allow perform external bash script to purge cache.
+Urls of attachments pass to script as arguments.
+
+* `script_path`: path to external script.
+
+Example:
+```elixir
+config :pleroma, Pleroma.Web.MediaProxy.Invalidation.Script,
+  script_path: "./installation/nginx-cache-purge.example"
+```
+
+#### Pleroma.Web.MediaProxy.Invalidation.Http
+
+This strategy allow perform custom http request to purge cache.
+
+* `method`: http method. default is `purge`
+* `headers`: http headers. default is empty
+* `options`: request options. default is empty
+
+Example:
+```elixir
+config :pleroma, Pleroma.Web.MediaProxy.Invalidation.Http,
+  method: :purge,
+  headers: [],
+  options: []
+```

 ## Link previews

@ -460,6 +503,7 @@ the source code is here: https://github.com/koto-bank/kocaptcha. The default end
 * `base_url`: The base URL to access a user-uploaded file. Useful when you want to proxy the media files via another host.
 * `proxy_remote`: If you're using a remote uploader, Pleroma will proxy media requests instead of redirecting to it.
 * `proxy_opts`: Proxy options, see `Pleroma.ReverseProxy` documentation.
+* `filename_display_max_length`: Set max length of a filename to display. 0 = no limit. Default: 30.

 !!! warning
    `strip_exif` has been replaced by `Pleroma.Upload.Filter.Mogrify`.
@ -620,24 +664,6 @@ config :pleroma, :workers,
 * `enabled: false` corresponds to `config :pleroma, :workers, retries: [federator_outgoing: 1]`
 * deprecated options: `max_jobs`, `initial_timeout`

-### Pleroma.Scheduler
-
-Configuration for [Quantum](https://github.com/quantum-elixir/quantum-core) jobs scheduler.
-
-See [Quantum readme](https://github.com/quantum-elixir/quantum-core#usage) for the list of supported options.
-
-Example:
-
-```elixir
-config :pleroma, Pleroma.Scheduler,
-  global: true,
-  overlap: true,
-  timezone: :utc,
-  jobs: [{"0 */6 * * * *", {Pleroma.Web.Websub, :refresh_subscriptions, []}}]
-```
-
-The above example defines a single job which invokes `Pleroma.Web.Websub.refresh_subscriptions()` every 6 hours ("0 */6 * * * *", [crontab format](https://en.wikipedia.org/wiki/Cron)).
-
 ## :web_push_encryption, :vapid_details

 Web Push Notifications configuration. You can use the mix task `mix web_push.gen.keypair` to generate it.
@ -908,12 +934,33 @@ config :auto_linker,

 * `runtime_dir`: A path to custom Elixir modules (such as MRF policies).

-
 ## :configurable_from_database

 Boolean, enables/disables in-database configuration. Read [Transfering the config to/from the database](../administration/CLI_tasks/config.md) for more information.

+## :database_config_whitelist

+List of valid configuration sections which are allowed to be configured from the
+database. Settings stored in the database before the whitelist is configured are
+still applied, so it is suggested to only use the whitelist on instances that
+have not migrated the config to the database.
+
+Example:
+```elixir
+config :pleroma, :database_config_whitelist, [
+  {:pleroma, :instance},
+  {:pleroma, Pleroma.Web.Metadata},
+  {:auto_linker}
+]
+```
+
+### Multi-factor authentication -  :two_factor_authentication
+* `totp` - a list containing TOTP configuration
+  - `digits` - Determines the length of a one-time pass-code in characters. Defaults to 6 characters.
+  - `period` - a period for which the TOTP code will be valid in seconds. Defaults to 30 seconds.
+* `backup_codes` - a list containing backup codes configuration
+  - `number` - number of backup codes to generate.
+  - `length` - backup code length. Defaults to 16 characters.

 ## Restrict entities access for unauthenticated users

@ -930,3 +977,8 @@ Restrict access for unauthenticated users to timelines (public and federate), us
 * `activities` - statuses
  * `local`
  * `remote`
+
+
+## Pleroma.Web.ApiSpec.CastAndValidate
+
+* `:strict` a boolean, enables strict input validation (useful in development, not recommended in production). Defaults to `false`.
--- a/docs/configuration/postgresql.md
+++ b/docs/configuration/postgresql.md
@ -0,0 +1,31 @@
+# Optimizing your PostgreSQL performance
+
+Pleroma performance depends to a large extent on good database performance. The default PostgreSQL settings are mostly fine, but often you can get better performance by changing a few settings.
+
+You can use [PGTune](https://pgtune.leopard.in.ua) to get recommendations for your setup. If you do, set the "Number of Connections" field to 20, as Pleroma will only use 10 concurrent connections anyway. If you don't, it will give you advice that might even hurt your performance.
+
+We also recommend not using the "Network Storage" option.
+
+## Example configurations
+
+Here are some configuration suggestions for PostgreSQL 10+.
+
+### 1GB RAM, 1 CPU
+```
+shared_buffers = 256MB
+effective_cache_size = 768MB
+maintenance_work_mem = 64MB
+work_mem = 13107kB
+```
+
+### 2GB RAM, 2 CPU
+```
+shared_buffers = 512MB
+effective_cache_size = 1536MB
+maintenance_work_mem = 128MB
+work_mem = 26214kB
+max_worker_processes = 2
+max_parallel_workers_per_gather = 1
+max_parallel_workers = 2
+```
+
--- a/docs/configuration/storing_remote_media.md
+++ b/docs/configuration/storing_remote_media.md
@ -0,0 +1,38 @@
+# Storing Remote Media
+
+Pleroma does not store remote/federated media by default. The best way to achieve this is to change Nginx to keep its reverse proxy cache
+for a year and to activate the `MediaProxyWarmingPolicy` MRF policy in Pleroma which will automatically fetch all media through the proxy
+as soon as the post is received by your instance.
+
+## Nginx
+
+```
+    proxy_cache_path /long/term/storage/path/pleroma-media-cache levels=1:2
+        keys_zone=pleroma_media_cache:10m inactive=1y use_temp_path=off;
+
+    location ~ ^/(media|proxy) {
+        proxy_cache        pleroma_media_cache;
+        slice              1m;
+        proxy_cache_key    $host$uri$is_args$args$slice_range;
+        proxy_set_header   Range $slice_range;
+        proxy_http_version 1.1;
+        proxy_cache_valid  206 301 302 304 1h;
+        proxy_cache_valid  200 1y;
+        proxy_cache_use_stale error timeout invalid_header updating;
+        proxy_ignore_client_abort on;
+        proxy_buffering    on;
+        chunked_transfer_encoding on;
+        proxy_ignore_headers Cache-Control Expires;
+        proxy_hide_header  Cache-Control Expires;
+        proxy_pass         http://127.0.0.1:4000;
+    }
+```
+
+## Pleroma
+
+Add to your `prod.secret.exs`:
+
+```
+config :pleroma, :instance,
+  rewrite_policy: [Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy]
+```
--- a/docs/installation/debian_based_en.md
+++ b/docs/installation/debian_based_en.md
@ -38,8 +38,8 @@ sudo apt install git build-essential postgresql postgresql-contrib
 * Download and add the Erlang repository:

 ```shell
-wget -P /tmp/ https://packages.erlang-solutions.com/erlang-solutions_1.0_all.deb
-sudo dpkg -i /tmp/erlang-solutions_1.0_all.deb
+wget -P /tmp/ https://packages.erlang-solutions.com/erlang-solutions_2.0_all.deb
+sudo dpkg -i /tmp/erlang-solutions_2.0_all.deb
 ```

 * Install Elixir and Erlang:
--- a/docs/installation/debian_based_jp.md
+++ b/docs/installation/debian_based_jp.md
@ -40,8 +40,8 @@ sudo apt install git build-essential postgresql postgresql-contrib

 * Erlangのリポジトリをダウンロードおよびインストールします。
 ```
-wget -P /tmp/ https://packages.erlang-solutions.com/erlang-solutions_1.0_all.deb
-sudo dpkg -i /tmp/erlang-solutions_1.0_all.deb
+wget -P /tmp/ https://packages.erlang-solutions.com/erlang-solutions_2.0_all.deb
+sudo dpkg -i /tmp/erlang-solutions_2.0_all.deb
 ```

 * ElixirとErlangをインストールします、
--- a/docs/installation/otp_en.md
+++ b/docs/installation/otp_en.md
@ -63,7 +63,7 @@ apt install postgresql-11-rum
 ```

 #### (Optional) Performance configuration
-For optimal performance, you may use [PGTune](https://pgtune.leopard.in.ua), don't forget to restart postgresql after editing the configuration
+It is encouraged to check [Optimizing your PostgreSQL performance](../configuration/postgresql.md) document, for tips on PostgreSQL tuning.

 ```sh tab="Alpine"
 rc-service postgresql restart
--- a/installation/init.d/pleroma
+++ b/installation/init.d/pleroma
@ -1,21 +1,45 @@
 #!/sbin/openrc-run
-
-# Requires OpenRC >= 0.35
-directory=/opt/pleroma
-
-command=/usr/bin/mix
-command_args="phx.server"
+supervisor=supervise-daemon
 command_user=pleroma:pleroma
 command_background=1
-
-export PORT=4000
-export MIX_ENV=prod
-
 # Ask process to terminate within 30 seconds, otherwise kill it
 retry="SIGTERM/30/SIGKILL/5"
-
 pidfile="/var/run/pleroma.pid"
+directory=/opt/pleroma
+healthcheck_delay=60
+healthcheck_timer=30
+
+: ${pleroma_port:-4000}
+
+# Needs OpenRC >= 0.42
+#respawn_max=0
+#respawn_delay=5
+
+# put pleroma_console=YES in /etc/conf.d/pleroma if you want to be able to
+# connect to pleroma via an elixir console
+if yesno "${pleroma_console}"; then
+	command=elixir
+	command_args="--name pleroma@127.0.0.1 --erl '-kernel inet_dist_listen_min 9001 inet_dist_listen_max 9001 inet_dist_use_interface {127,0,0,1}' -S mix phx.server"
+
+	start_post() {
+		einfo "You can get a console by using this command as pleroma's user:"
+		einfo "iex --name console@127.0.0.1 --remsh pleroma@127.0.0.1"
+	}
+else
+	command=/usr/bin/mix
+	command_args="phx.server"
+fi
+
+export MIX_ENV=prod

 depend() {
-    need nginx postgresql
+	need nginx postgresql
+}
+
+healthcheck() {
+	# put pleroma_health=YES in /etc/conf.d/pleroma if you want healthchecking
+	# and make sure you have curl installed
+	yesno "$pleroma_health" || return 0
+
+	curl -q "localhost:${pleroma_port}/api/pleroma/healthcheck"
 }
--- a/installation/nginx-cache-purge.sh.example
+++ b/installation/nginx-cache-purge.sh.example
@ -0,0 +1,40 @@
+#!/bin/sh
+
+# A simple shell script to delete a media from the Nginx cache.
+
+SCRIPTNAME=${0##*/}
+
+# NGINX cache directory
+CACHE_DIRECTORY="/tmp/pleroma-media-cache"
+
+## Return the files where the items are cached.
+## $1 - the filename, can be a pattern .
+## $2 - the cache directory.
+## $3 - (optional) the number of parallel processes to run for grep.
+get_cache_files() {
+    local max_parallel=${3-16}
+    find $2 -maxdepth 2 -type d | xargs -P $max_parallel -n 1 grep -E Rl "^KEY:.*$1" | sort -u
+}
+
+## Removes an item from the given cache zone.
+## $1 - the filename, can be a pattern .
+## $2 - the cache directory.
+purge_item() {
+  for f in $(get_cache_files $1 $2); do
+      echo "found file: $f"
+      [ -f $f ] || continue
+      echo "Deleting $f from $2."
+      rm $f
+  done
+} # purge_item
+
+purge() {
+  for url in "$@"
+  do
+    echo "$SCRIPTNAME delete \`$url\` from cache ($CACHE_DIRECTORY)"
+    purge_item $url $CACHE_DIRECTORY
+  done
+
+}
+
+purge $1
--- a/installation/pleroma-apache.conf
+++ b/installation/pleroma-apache.conf
@ -32,9 +32,8 @@ CustomLog ${APACHE_LOG_DIR}/access.log combined

 <VirtualHost *:443>
    SSLEngine on
-    SSLCertificateFile      /etc/letsencrypt/live/${servername}/cert.pem
+    SSLCertificateFile      /etc/letsencrypt/live/${servername}/fullchain.pem
    SSLCertificateKeyFile   /etc/letsencrypt/live/${servername}/privkey.pem
-    SSLCertificateChainFile /etc/letsencrypt/live/${servername}/fullchain.pem

    # Mozilla modern configuration, tweak to your needs
    SSLProtocol             all -SSLv3 -TLSv1 -TLSv1.1
--- a/lib/mix/tasks/pleroma/benchmark.ex
+++ b/lib/mix/tasks/pleroma/benchmark.ex
@ -67,8 +67,7 @@ def run(["render_timeline", nickname | _] = args) do
          Pleroma.Web.MastodonAPI.StatusView.render("index.json", %{
            activities: activities,
            for: user,
-            as: :activity,
-            skip_relationships: true
+            as: :activity
          })
        end
      },
--- a/lib/mix/tasks/pleroma/database.ex
+++ b/lib/mix/tasks/pleroma/database.ex
@ -4,6 +4,7 @@

 defmodule Mix.Tasks.Pleroma.Database do
  alias Pleroma.Conversation
+  alias Pleroma.Maintenance
  alias Pleroma.Object
  alias Pleroma.Repo
  alias Pleroma.User
@ -34,13 +35,7 @@ def run(["remove_embedded_objects" | args]) do
    )

    if Keyword.get(options, :vacuum) do
-      Logger.info("Runnning VACUUM FULL")
-
-      Repo.query!(
-        "vacuum full;",
-        [],
-        timeout: :infinity
-      )
+      Maintenance.vacuum("full")
    end
  end

@ -94,13 +89,7 @@ def run(["prune_objects" | args]) do
    |> Repo.delete_all(timeout: :infinity)

    if Keyword.get(options, :vacuum) do
-      Logger.info("Runnning VACUUM FULL")
-
-      Repo.query!(
-        "vacuum full;",
-        [],
-        timeout: :infinity
-      )
+      Maintenance.vacuum("full")
    end
  end

@ -135,4 +124,10 @@ def run(["fix_likes_collections"]) do
    end)
    |> Stream.run()
  end
+
+  def run(["vacuum", args]) do
+    start_pleroma()
+
+    Maintenance.vacuum(args)
+  end
 end
--- a/lib/mix/tasks/pleroma/digest.ex
+++ b/lib/mix/tasks/pleroma/digest.ex
@ -1,5 +1,6 @@
 defmodule Mix.Tasks.Pleroma.Digest do
  use Mix.Task
+  import Mix.Pleroma

  @shortdoc "Manages digest emails"
  @moduledoc File.read!("docs/administration/CLI_tasks/digest.md")
@ -22,12 +23,10 @@ def run(["test", nickname | opts]) do
    with %Swoosh.Email{} = email <- Pleroma.Emails.UserEmail.digest_email(patched_user) do
      {:ok, _} = Pleroma.Emails.Mailer.deliver(email)

-      Mix.shell().info("Digest email have been sent to #{nickname} (#{user.email})")
+      shell_info("Digest email have been sent to #{nickname} (#{user.email})")
    else
      _ ->
-        Mix.shell().info(
-          "Cound't find any mentions for #{nickname} since #{last_digest_emailed_at}"
-        )
+        shell_info("Cound't find any mentions for #{nickname} since #{last_digest_emailed_at}")
    end
  end
 end
--- a/lib/mix/tasks/pleroma/emoji.ex
+++ b/lib/mix/tasks/pleroma/emoji.ex
@ -15,7 +15,7 @@ def run(["ls-packs" | args]) do
    {options, [], []} = parse_global_opts(args)

    url_or_path = options[:manifest] || default_manifest()
-    manifest = fetch_manifest(url_or_path)
+    manifest = fetch_and_decode(url_or_path)

    Enum.each(manifest, fn {name, info} ->
      to_print = [
@ -42,12 +42,12 @@ def run(["get-packs" | args]) do

    url_or_path = options[:manifest] || default_manifest()

-    manifest = fetch_manifest(url_or_path)
+    manifest = fetch_and_decode(url_or_path)

    for pack_name <- pack_names do
      if Map.has_key?(manifest, pack_name) do
        pack = manifest[pack_name]
-        src_url = pack["src"]
+        src = pack["src"]

        IO.puts(
          IO.ANSI.format([
@ -57,11 +57,11 @@ def run(["get-packs" | args]) do
            :normal,
            " from ",
            :underline,
-            src_url
+            src
          ])
        )

-        binary_archive = Tesla.get!(client(), src_url).body
+        {:ok, binary_archive} = fetch(src)
        archive_sha = :crypto.hash(:sha256, binary_archive) |> Base.encode16()

        sha_status_text = ["SHA256 of ", :bright, pack_name, :normal, " source file is ", :bright]
@ -74,8 +74,8 @@ def run(["get-packs" | args]) do
          raise "Bad SHA256 for #{pack_name}"
        end

-        # The url specified in files should be in the same directory
-        files_url =
+        # The location specified in files should be in the same directory
+        files_loc =
          url_or_path
          |> Path.dirname()
          |> Path.join(pack["files"])
@ -88,11 +88,11 @@ def run(["get-packs" | args]) do
            :normal,
            " from ",
            :underline,
-            files_url
+            files_loc
          ])
        )

-        files = Tesla.get!(client(), files_url).body |> Jason.decode!()
+        files = fetch_and_decode(files_loc)

        IO.puts(IO.ANSI.format(["Unpacking ", :bright, pack_name]))

@ -237,16 +237,20 @@ def run(["gen-pack" | args]) do
    end
  end

-  defp fetch_manifest(from) do
-    Jason.decode!(
-      if String.starts_with?(from, "http") do
-        Tesla.get!(client(), from).body
-      else
-        File.read!(from)
-      end
-    )
+  defp fetch_and_decode(from) do
+    with {:ok, json} <- fetch(from) do
+      Jason.decode!(json)
+    end
  end

+  defp fetch("http" <> _ = from) do
+    with {:ok, %{body: body}} <- Tesla.get(client(), from) do
+      {:ok, body}
+    end
+  end
+
+  defp fetch(path), do: File.read(path)
+
  defp parse_global_opts(args) do
    OptionParser.parse(
      args,
--- a/lib/mix/tasks/pleroma/instance.ex
+++ b/lib/mix/tasks/pleroma/instance.ex
@ -147,6 +147,7 @@ def run(["gen" | rest]) do
          "What directory should media uploads go in (when using the local uploader)?",
          Pleroma.Config.get([Pleroma.Uploaders.Local, :uploads])
        )
+        |> Path.expand()

      static_dir =
        get_option(
@ -155,6 +156,7 @@ def run(["gen" | rest]) do
          "What directory should custom public files be read from (custom emojis, frontend bundle overrides, robots.txt, etc.)?",
          Pleroma.Config.get([:instance, :static_dir])
        )
+        |> Path.expand()

      Config.put([:instance, :static_dir], static_dir)

@ -204,7 +206,7 @@ def run(["gen" | rest]) do
      shell_info("Writing the postgres script to #{psql_path}.")
      File.write(psql_path, result_psql)

-      write_robots_txt(indexable, template_dir)
+      write_robots_txt(static_dir, indexable, template_dir)

      shell_info(
        "\n All files successfully written! Refer to the installation instructions for your platform for next steps."
@ -224,15 +226,13 @@ def run(["gen" | rest]) do
    end
  end

-  defp write_robots_txt(indexable, template_dir) do
+  defp write_robots_txt(static_dir, indexable, template_dir) do
    robots_txt =
      EEx.eval_file(
        template_dir <> "/robots_txt.eex",
        indexable: indexable
      )

-    static_dir = Pleroma.Config.get([:instance, :static_dir], "instance/static/")
-
    unless File.exists?(static_dir) do
      File.mkdir_p!(static_dir)
    end
--- a/lib/mix/tasks/pleroma/user.ex
+++ b/lib/mix/tasks/pleroma/user.ex
@ -8,6 +8,8 @@ defmodule Mix.Tasks.Pleroma.User do
  alias Ecto.Changeset
  alias Pleroma.User
  alias Pleroma.UserInviteToken
+  alias Pleroma.Web.ActivityPub.Builder
+  alias Pleroma.Web.ActivityPub.Pipeline

  @shortdoc "Manages Pleroma users"
  @moduledoc File.read!("docs/administration/CLI_tasks/user.md")
@ -96,8 +98,9 @@ def run(["new", nickname, email | rest]) do
  def run(["rm", nickname]) do
    start_pleroma()

-    with %User{local: true} = user <- User.get_cached_by_nickname(nickname) do
-      User.perform(:delete, user)
+    with %User{local: true} = user <- User.get_cached_by_nickname(nickname),
+         {:ok, delete_data, _} <- Builder.delete(user, user.ap_id),
+         {:ok, _delete, _} <- Pipeline.common_pipeline(delete_data, local: true) do
      shell_info("User #{nickname} deleted.")
    else
      _ -> shell_error("No local user #{nickname}")
@ -141,28 +144,18 @@ def run(["reset_password", nickname]) do
    end
  end

-  def run(["unsubscribe", nickname]) do
+  def run(["deactivate", nickname]) do
    start_pleroma()

    with %User{} = user <- User.get_cached_by_nickname(nickname) do
      shell_info("Deactivating #{user.nickname}")
      User.deactivate(user)
-
-      user
-      |> User.get_friends()
-      |> Enum.each(fn friend ->
-        user = User.get_cached_by_id(user.id)
-
-        shell_info("Unsubscribing #{friend.nickname} from #{user.nickname}")
-        User.unfollow(user, friend)
-      end)
-
      :timer.sleep(500)

      user = User.get_cached_by_id(user.id)

-      if Enum.empty?(User.get_friends(user)) do
-        shell_info("Successfully unsubscribed all followers from #{user.nickname}")
+      if Enum.empty?(Enum.filter(User.get_friends(user), & &1.local)) do
+        shell_info("Successfully unsubscribed all local followers from #{user.nickname}")
      end
    else
      _ ->
@ -170,7 +163,7 @@ def run(["unsubscribe", nickname]) do
    end
  end

-  def run(["unsubscribe_all_from_instance", instance]) do
+  def run(["deactivate_all_from_instance", instance]) do
    start_pleroma()

    Pleroma.User.Query.build(%{nickname: "@#{instance}"})
@ -178,7 +171,7 @@ def run(["unsubscribe_all_from_instance", instance]) do
    |> Stream.each(fn users ->
      users
      |> Enum.each(fn user ->
-        run(["unsubscribe", user.nickname])
+        run(["deactivate", user.nickname])
      end)
    end)
    |> Stream.run()
--- a/lib/pleroma/activity/queries.ex
+++ b/lib/pleroma/activity/queries.ex
@ -24,10 +24,7 @@ def by_ap_id(query \\ Activity, ap_id) do

  @spec by_actor(query, String.t()) :: query
  def by_actor(query \\ Activity, actor) do
-    from(
-      activity in query,
-      where: fragment("(?)->>'actor' = ?", activity.data, ^actor)
-    )
+    from(a in query, where: a.actor == ^actor)
  end

  @spec by_author(query, User.t()) :: query
--- a/lib/pleroma/application.ex
+++ b/lib/pleroma/application.ex
@ -56,7 +56,7 @@ def start(_type, _args) do
        if (major == 22 and minor < 2) or major < 22 do
          raise "
            !!!OTP VERSION WARNING!!!
-            You are using gun adapter with OTP version #{version}, which doesn't support correct handling of unordered certificates chains.
+            You are using gun adapter with OTP version #{version}, which doesn't support correct handling of unordered certificates chains. Please update your Erlang/OTP to at least 22.2.
            "
        end
      else
@ -173,7 +173,14 @@ defp chat_enabled?, do: Config.get([:chat, :enabled])
  defp streamer_child(env) when env in [:test, :benchmark], do: []

  defp streamer_child(_) do
-    [Pleroma.Web.Streamer.supervisor()]
+    [
+      {Registry,
+       [
+         name: Pleroma.Web.Streamer.registry(),
+         keys: :duplicate,
+         partitions: System.schedulers_online()
+       ]}
+    ]
  end

  defp chat_child(_env, true) do
--- a/lib/pleroma/bbs/authenticator.ex
+++ b/lib/pleroma/bbs/authenticator.ex
@ -4,7 +4,7 @@

 defmodule Pleroma.BBS.Authenticator do
  use Sshd.PasswordAuthenticator
-  alias Comeonin.Pbkdf2
+  alias Pleroma.Plugs.AuthenticationPlug
  alias Pleroma.User

  def authenticate(username, password) do
@ -12,7 +12,7 @@ def authenticate(username, password) do
    password = to_string(password)

    with %User{} = user <- User.get_by_nickname(username) do
-      Pbkdf2.checkpw(password, user.password_hash)
+      AuthenticationPlug.checkpw(password, user.password_hash)
    else
      _e -> false
    end
--- a/lib/pleroma/bbs/handler.ex
+++ b/lib/pleroma/bbs/handler.ex
@ -66,7 +66,7 @@ def handle_command(%{user: user} = state, "r " <> text) do

    with %Activity{} <- Activity.get_by_id(activity_id),
         {:ok, _activity} <-
-           CommonAPI.post(user, %{"status" => rest, "in_reply_to_status_id" => activity_id}) do
+           CommonAPI.post(user, %{status: rest, in_reply_to_status_id: activity_id}) do
      IO.puts("Replied!")
    else
      _e -> IO.puts("Could not reply...")
@ -78,7 +78,7 @@ def handle_command(%{user: user} = state, "r " <> text) do
  def handle_command(%{user: user} = state, "p " <> text) do
    text = String.trim(text)

-    with {:ok, _activity} <- CommonAPI.post(user, %{"status" => text}) do
+    with {:ok, _activity} <- CommonAPI.post(user, %{status: text}) do
      IO.puts("Posted!")
    else
      _e -> IO.puts("Could not post...")
--- a/lib/pleroma/config/config_db.ex
+++ b/lib/pleroma/config/config_db.ex
@ -278,6 +278,8 @@ defp do_convert({:proxy_url, {type, host, port}}) do
    }
  end

+  defp do_convert({:partial_chain, entity}), do: %{"tuple" => [":partial_chain", inspect(entity)]}
+
  defp do_convert(entity) when is_tuple(entity) do
    value =
      entity
@ -321,6 +323,15 @@ defp do_transform(%{"tuple" => [":proxy_url", %{"tuple" => [type, host, port]}]}
    {:proxy_url, {do_transform_string(type), parse_host(host), port}}
  end

+  defp do_transform(%{"tuple" => [":partial_chain", entity]}) do
+    {partial_chain, []} =
+      entity
+      |> String.replace(~r/[^\w|^{:,[|^,|^[|^\]^}|^\/|^\.|^"]^\s/, "")
+      |> Code.eval_string()
+
+    {:partial_chain, partial_chain}
+  end
+
  defp do_transform(%{"tuple" => entity}) do
    Enum.reduce(entity, {}, fn val, acc -> Tuple.append(acc, do_transform(val)) end)
  end
--- a/lib/pleroma/constants.ex
+++ b/lib/pleroma/constants.ex
@ -17,7 +17,8 @@ defmodule Pleroma.Constants do
      "announcement_count",
      "emoji",
      "context_id",
-      "deleted_activity_id"
+      "deleted_activity_id",
+      "pleroma_internal"
    ]
  )

--- a/lib/pleroma/conversation.ex
+++ b/lib/pleroma/conversation.ex
@ -63,7 +63,7 @@ def create_or_bump_for(activity, opts \\ []) do
         ap_id when is_binary(ap_id) and byte_size(ap_id) > 0 <- object.data["context"] do
      {:ok, conversation} = create_for_ap_id(ap_id)

-      users = User.get_users_from_set(activity.recipients, false)
+      users = User.get_users_from_set(activity.recipients, local_only: false)

      participations =
        Enum.map(users, fn user ->
--- a/lib/pleroma/conversation/participation.ex
+++ b/lib/pleroma/conversation/participation.ex
@ -128,7 +128,7 @@ def for_user(user, params \\ %{}) do
    |> Pleroma.Pagination.fetch_paginated(params)
  end

-  def restrict_recipients(query, user, %{"recipients" => user_ids}) do
+  def restrict_recipients(query, user, %{recipients: user_ids}) do
    user_binary_ids =
      [user.id | user_ids]
      |> Enum.uniq()
@ -172,7 +172,7 @@ def for_user_with_last_activity_id(user, params \\ %{}) do
        | last_activity_id: activity_id
      }
    end)
-    |> Enum.filter(& &1.last_activity_id)
+    |> Enum.reject(&is_nil(&1.last_activity_id))
  end

  def get(_, _ \\ [])
--- a/lib/pleroma/docs/json.ex
+++ b/lib/pleroma/docs/json.ex
@ -18,7 +18,6 @@ def compile do
    with config <- Pleroma.Config.Loader.read("config/description.exs") do
      config[:pleroma][:config_description]
      |> Pleroma.Docs.Generator.convert_to_strings()
-      |> Jason.encode!()
    end
  end
 end
--- a/lib/pleroma/emails/new_users_digest_email.ex
+++ b/lib/pleroma/emails/new_users_digest_email.ex
@ -14,8 +14,10 @@ def new_users(to, users_and_statuses) do
    styling = Pleroma.Config.get([Pleroma.Emails.UserEmail, :styling])

    logo_url =
-      Pleroma.Web.Endpoint.url() <>
-        Pleroma.Config.get([:frontend_configurations, :pleroma_fe, :logo])
+      Pleroma.Helpers.UriHelper.maybe_add_base(
+        Pleroma.Config.get([:frontend_configurations, :pleroma_fe, :logo]),
+        Pleroma.Web.Endpoint.url()
+      )

    new()
    |> to({to.name, to.email})
--- a/lib/pleroma/emoji/pack.ex
+++ b/lib/pleroma/emoji/pack.ex
@ -16,162 +16,78 @@ defmodule Pleroma.Emoji.Pack do

  alias Pleroma.Emoji

-  @spec emoji_path() :: Path.t()
-  def emoji_path do
-    static = Pleroma.Config.get!([:instance, :static_dir])
-    Path.join(static, "emoji")
-  end
-
  @spec create(String.t()) :: :ok | {:error, File.posix()} | {:error, :empty_values}
-  def create(name) when byte_size(name) > 0 do
-    dir = Path.join(emoji_path(), name)
-
-    with :ok <- File.mkdir(dir) do
-      %__MODULE__{
-        pack_file: Path.join(dir, "pack.json")
-      }
+  def create(name) do
+    with :ok <- validate_not_empty([name]),
+         dir <- Path.join(emoji_path(), name),
+         :ok <- File.mkdir(dir) do
+      %__MODULE__{pack_file: Path.join(dir, "pack.json")}
      |> save_pack()
    end
  end

-  def create(_), do: {:error, :empty_values}
-
-  @spec show(String.t()) :: {:ok, t()} | {:loaded, nil} | {:error, :empty_values}
-  def show(name) when byte_size(name) > 0 do
-    with {_, %__MODULE__{} = pack} <- {:loaded, load_pack(name)},
-         {_, pack} <- validate_pack(pack) do
-      {:ok, pack}
+  @spec show(String.t()) :: {:ok, t()} | {:error, atom()}
+  def show(name) do
+    with :ok <- validate_not_empty([name]),
+         {:ok, pack} <- load_pack(name) do
+      {:ok, validate_pack(pack)}
    end
  end

-  def show(_), do: {:error, :empty_values}
-
  @spec delete(String.t()) ::
          {:ok, [binary()]} | {:error, File.posix(), binary()} | {:error, :empty_values}
-  def delete(name) when byte_size(name) > 0 do
-    emoji_path()
-    |> Path.join(name)
-    |> File.rm_rf()
-  end
-
-  def delete(_), do: {:error, :empty_values}
-
-  @spec add_file(String.t(), String.t(), Path.t(), Plug.Upload.t() | String.t()) ::
-          {:ok, t()} | {:error, File.posix()} | {:error, :empty_values}
-  def add_file(name, shortcode, filename, file)
-      when byte_size(name) > 0 and byte_size(shortcode) > 0 and byte_size(filename) > 0 do
-    with {_, nil} <- {:exists, Emoji.get(shortcode)},
-         {_, %__MODULE__{} = pack} <- {:loaded, load_pack(name)} do
-      file_path = Path.join(pack.path, filename)
-
-      create_subdirs(file_path)
-
-      case file do
-        %Plug.Upload{path: upload_path} ->
-          # Copy the uploaded file from the temporary directory
-          File.copy!(upload_path, file_path)
-
-        url when is_binary(url) ->
-          # Download and write the file
-          file_contents = Tesla.get!(url).body
-          File.write!(file_path, file_contents)
-      end
-
-      files = Map.put(pack.files, shortcode, filename)
-
-      updated_pack = %{pack | files: files}
-
-      case save_pack(updated_pack) do
-        :ok ->
-          Emoji.reload()
-          {:ok, updated_pack}
-
-        e ->
-          e
-      end
+  def delete(name) do
+    with :ok <- validate_not_empty([name]) do
+      emoji_path()
+      |> Path.join(name)
+      |> File.rm_rf()
    end
  end

-  def add_file(_, _, _, _), do: {:error, :empty_values}
-
-  defp create_subdirs(file_path) do
-    if String.contains?(file_path, "/") do
-      file_path
-      |> Path.dirname()
-      |> File.mkdir_p!()
+  @spec add_file(String.t(), String.t(), Path.t(), Plug.Upload.t() | String.t()) ::
+          {:ok, t()} | {:error, File.posix() | atom()}
+  def add_file(name, shortcode, filename, file) do
+    with :ok <- validate_not_empty([name, shortcode, filename]),
+         :ok <- validate_emoji_not_exists(shortcode),
+         {:ok, pack} <- load_pack(name),
+         :ok <- save_file(file, pack, filename),
+         {:ok, updated_pack} <- pack |> put_emoji(shortcode, filename) |> save_pack() do
+      Emoji.reload()
+      {:ok, updated_pack}
    end
  end

  @spec delete_file(String.t(), String.t()) ::
-          {:ok, t()} | {:error, File.posix()} | {:error, :empty_values}
-  def delete_file(name, shortcode) when byte_size(name) > 0 and byte_size(shortcode) > 0 do
-    with {_, %__MODULE__{} = pack} <- {:loaded, load_pack(name)},
-         {_, {filename, files}} when not is_nil(filename) <-
-           {:exists, Map.pop(pack.files, shortcode)},
-         emoji <- Path.join(pack.path, filename),
-         {_, true} <- {:exists, File.exists?(emoji)} do
-      emoji_dir = Path.dirname(emoji)
-
-      File.rm!(emoji)
-
-      if String.contains?(filename, "/") and File.ls!(emoji_dir) == [] do
-        File.rmdir!(emoji_dir)
-      end
-
-      updated_pack = %{pack | files: files}
-
-      case save_pack(updated_pack) do
-        :ok ->
-          Emoji.reload()
-          {:ok, updated_pack}
-
-        e ->
-          e
-      end
+          {:ok, t()} | {:error, File.posix() | atom()}
+  def delete_file(name, shortcode) do
+    with :ok <- validate_not_empty([name, shortcode]),
+         {:ok, pack} <- load_pack(name),
+         :ok <- remove_file(pack, shortcode),
+         {:ok, updated_pack} <- pack |> delete_emoji(shortcode) |> save_pack() do
+      Emoji.reload()
+      {:ok, updated_pack}
    end
  end

-  def delete_file(_, _), do: {:error, :empty_values}
-
  @spec update_file(String.t(), String.t(), String.t(), String.t(), boolean()) ::
-          {:ok, t()} | {:error, File.posix()} | {:error, :empty_values}
-  def update_file(name, shortcode, new_shortcode, new_filename, force)
-      when byte_size(name) > 0 and byte_size(shortcode) > 0 and byte_size(new_shortcode) > 0 and
-             byte_size(new_filename) > 0 do
-    with {_, %__MODULE__{} = pack} <- {:loaded, load_pack(name)},
-         {_, {filename, files}} when not is_nil(filename) <-
-           {:exists, Map.pop(pack.files, shortcode)},
-         {_, true} <- {:not_used, force or is_nil(Emoji.get(new_shortcode))} do
-      old_path = Path.join(pack.path, filename)
-      old_dir = Path.dirname(old_path)
-      new_path = Path.join(pack.path, new_filename)
-
-      create_subdirs(new_path)
-
-      :ok = File.rename(old_path, new_path)
-
-      if String.contains?(filename, "/") and File.ls!(old_dir) == [] do
-        File.rmdir!(old_dir)
-      end
-
-      files = Map.put(files, new_shortcode, new_filename)
-
-      updated_pack = %{pack | files: files}
-
-      case save_pack(updated_pack) do
-        :ok ->
-          Emoji.reload()
-          {:ok, updated_pack}
-
-        e ->
-          e
-      end
+          {:ok, t()} | {:error, File.posix() | atom()}
+  def update_file(name, shortcode, new_shortcode, new_filename, force) do
+    with :ok <- validate_not_empty([name, shortcode, new_shortcode, new_filename]),
+         {:ok, pack} <- load_pack(name),
+         {:ok, filename} <- get_filename(pack, shortcode),
+         :ok <- validate_emoji_not_exists(new_shortcode, force),
+         :ok <- rename_file(pack, filename, new_filename),
+         {:ok, updated_pack} <-
+           pack
+           |> delete_emoji(shortcode)
+           |> put_emoji(new_shortcode, new_filename)
+           |> save_pack() do
+      Emoji.reload()
+      {:ok, updated_pack}
    end
  end

-  def update_file(_, _, _, _, _), do: {:error, :empty_values}
-
-  @spec import_from_filesystem() :: {:ok, [String.t()]} | {:error, atom()}
+  @spec import_from_filesystem() :: {:ok, [String.t()]} | {:error, File.posix() | atom()}
  def import_from_filesystem do
    emoji_path = emoji_path()

@ -184,7 +100,7 @@ def import_from_filesystem do
          File.dir?(path) and File.exists?(Path.join(path, "pack.json"))
        end)
        |> Enum.map(&write_pack_contents/1)
-        |> Enum.filter(& &1)
+        |> Enum.reject(&is_nil/1)

      {:ok, names}
    else
@ -193,6 +109,117 @@ def import_from_filesystem do
    end
  end

+  @spec list_remote(String.t()) :: {:ok, map()} | {:error, atom()}
+  def list_remote(url) do
+    uri = url |> String.trim() |> URI.parse()
+
+    with :ok <- validate_shareable_packs_available(uri) do
+      uri
+      |> URI.merge("/api/pleroma/emoji/packs")
+      |> http_get()
+    end
+  end
+
+  @spec list_local() :: {:ok, map()}
+  def list_local do
+    with {:ok, results} <- list_packs_dir() do
+      packs =
+        results
+        |> Enum.map(fn name ->
+          case load_pack(name) do
+            {:ok, pack} -> pack
+            _ -> nil
+          end
+        end)
+        |> Enum.reject(&is_nil/1)
+        |> Map.new(fn pack -> {pack.name, validate_pack(pack)} end)
+
+      {:ok, packs}
+    end
+  end
+
+  @spec get_archive(String.t()) :: {:ok, binary()} | {:error, atom()}
+  def get_archive(name) do
+    with {:ok, pack} <- load_pack(name),
+         :ok <- validate_downloadable(pack) do
+      {:ok, fetch_archive(pack)}
+    end
+  end
+
+  @spec download(String.t(), String.t(), String.t()) :: :ok | {:error, atom()}
+  def download(name, url, as) do
+    uri = url |> String.trim() |> URI.parse()
+
+    with :ok <- validate_shareable_packs_available(uri),
+         {:ok, remote_pack} <- uri |> URI.merge("/api/pleroma/emoji/packs/#{name}") |> http_get(),
+         {:ok, %{sha: sha, url: url} = pack_info} <- fetch_pack_info(remote_pack, uri, name),
+         {:ok, archive} <- download_archive(url, sha),
+         pack <- copy_as(remote_pack, as || name),
+         {:ok, _} = unzip(archive, pack_info, remote_pack, pack) do
+      # Fallback can't contain a pack.json file, since that would cause the fallback-src-sha256
+      # in it to depend on itself
+      if pack_info[:fallback] do
+        save_pack(pack)
+      else
+        {:ok, pack}
+      end
+    end
+  end
+
+  @spec save_metadata(map(), t()) :: {:ok, t()} | {:error, File.posix()}
+  def save_metadata(metadata, %__MODULE__{} = pack) do
+    pack
+    |> Map.put(:pack, metadata)
+    |> save_pack()
+  end
+
+  @spec update_metadata(String.t(), map()) :: {:ok, t()} | {:error, File.posix()}
+  def update_metadata(name, data) do
+    with {:ok, pack} <- load_pack(name) do
+      if fallback_sha_changed?(pack, data) do
+        update_sha_and_save_metadata(pack, data)
+      else
+        save_metadata(data, pack)
+      end
+    end
+  end
+
+  @spec load_pack(String.t()) :: {:ok, t()} | {:error, :not_found}
+  def load_pack(name) do
+    pack_file = Path.join([emoji_path(), name, "pack.json"])
+
+    if File.exists?(pack_file) do
+      pack =
+        pack_file
+        |> File.read!()
+        |> from_json()
+        |> Map.put(:pack_file, pack_file)
+        |> Map.put(:path, Path.dirname(pack_file))
+        |> Map.put(:name, name)
+
+      {:ok, pack}
+    else
+      {:error, :not_found}
+    end
+  end
+
+  @spec emoji_path() :: Path.t()
+  defp emoji_path do
+    [:instance, :static_dir]
+    |> Pleroma.Config.get!()
+    |> Path.join("emoji")
+  end
+
+  defp validate_emoji_not_exists(shortcode, force \\ false)
+  defp validate_emoji_not_exists(_shortcode, true), do: :ok
+
+  defp validate_emoji_not_exists(shortcode, _) do
+    case Emoji.get(shortcode) do
+      nil -> :ok
+      _ -> {:error, :already_exists}
+    end
+  end
+
  defp write_pack_contents(path) do
    pack = %__MODULE__{
      files: files_from_path(path),
@ -201,7 +228,7 @@ defp write_pack_contents(path) do
    }

    case save_pack(pack) do
-      :ok -> Path.basename(path)
+      {:ok, _pack} -> Path.basename(path)
      _ -> nil
    end
  end
@ -216,7 +243,8 @@ defp files_from_path(path) do
      # FIXME: Copy-pasted from Pleroma.Emoji/load_from_file_stream/2

      # Create a map of shortcodes to filenames from emoji.txt
-      File.read!(txt_path)
+      txt_path
+      |> File.read!()
      |> String.split("\n")
      |> Enum.map(&String.trim/1)
      |> Enum.map(fn line ->
@ -226,21 +254,18 @@ defp files_from_path(path) do
          [name, file | _] ->
            file_dir_name = Path.dirname(file)

-            file =
-              if String.ends_with?(path, file_dir_name) do
-                Path.basename(file)
-              else
-                file
-              end
-
-            {name, file}
+            if String.ends_with?(path, file_dir_name) do
+              {name, Path.basename(file)}
+            else
+              {name, file}
+            end

          _ ->
            nil
        end
      end)
-      |> Enum.filter(& &1)
-      |> Enum.into(%{})
+      |> Enum.reject(&is_nil/1)
+      |> Map.new()
    else
      # If there's no emoji.txt, assume all files
      # that are of certain extensions from the config are emojis and import them all
@ -249,60 +274,20 @@ defp files_from_path(path) do
    end
  end

-  @spec list_remote(String.t()) :: {:ok, map()}
-  def list_remote(url) do
-    uri =
-      url
-      |> String.trim()
-      |> URI.parse()
-
-    with {_, true} <- {:shareable, shareable_packs_available?(uri)} do
-      packs =
-        uri
-        |> URI.merge("/api/pleroma/emoji/packs")
-        |> to_string()
-        |> Tesla.get!()
-        |> Map.get(:body)
-        |> Jason.decode!()
-
-      {:ok, packs}
-    end
-  end
-
-  @spec list_local() :: {:ok, map()}
-  def list_local do
-    emoji_path = emoji_path()
-
-    # Create the directory first if it does not exist. This is probably the first request made
-    # with the API so it should be sufficient
-    with {:create_dir, :ok} <- {:create_dir, File.mkdir_p(emoji_path)},
-         {:ls, {:ok, results}} <- {:ls, File.ls(emoji_path)} do
-      packs =
-        results
-        |> Enum.map(&load_pack/1)
-        |> Enum.filter(& &1)
-        |> Enum.map(&validate_pack/1)
-        |> Map.new()
-
-      {:ok, packs}
-    end
-  end
-
  defp validate_pack(pack) do
-    if downloadable?(pack) do
-      archive = fetch_archive(pack)
-      archive_sha = :crypto.hash(:sha256, archive) |> Base.encode16()
+    info =
+      if downloadable?(pack) do
+        archive = fetch_archive(pack)
+        archive_sha = :crypto.hash(:sha256, archive) |> Base.encode16()

-      info =
        pack.pack
        |> Map.put("can-download", true)
        |> Map.put("download-sha256", archive_sha)
+      else
+        Map.put(pack.pack, "can-download", false)
+      end

-      {pack.name, Map.put(pack, :pack, info)}
-    else
-      info = Map.put(pack.pack, "can-download", false)
-      {pack.name, Map.put(pack, :pack, info)}
-    end
+    Map.put(pack, :pack, info)
  end

  defp downloadable?(pack) do
@ -315,26 +300,6 @@ defp downloadable?(pack) do
      end)
  end

-  @spec get_archive(String.t()) :: {:ok, binary()}
-  def get_archive(name) do
-    with {_, %__MODULE__{} = pack} <- {:exists?, load_pack(name)},
-         {_, true} <- {:can_download?, downloadable?(pack)} do
-      {:ok, fetch_archive(pack)}
-    end
-  end
-
-  defp fetch_archive(pack) do
-    hash = :crypto.hash(:md5, File.read!(pack.pack_file))
-
-    case Cachex.get!(:emoji_packs_cache, pack.name) do
-      %{hash: ^hash, pack_data: archive} ->
-        archive
-
-      _ ->
-        create_archive_and_cache(pack, hash)
-    end
-  end
-
  defp create_archive_and_cache(pack, hash) do
    files = ['pack.json' | Enum.map(pack.files, fn {_, file} -> to_charlist(file) end)]

@ -356,152 +321,221 @@ defp create_archive_and_cache(pack, hash) do
    result
  end

-  @spec download(String.t(), String.t(), String.t()) :: :ok
-  def download(name, url, as) do
-    uri =
-      url
-      |> String.trim()
-      |> URI.parse()
-
-    with {_, true} <- {:shareable, shareable_packs_available?(uri)} do
-      remote_pack =
-        uri
-        |> URI.merge("/api/pleroma/emoji/packs/#{name}")
-        |> to_string()
-        |> Tesla.get!()
-        |> Map.get(:body)
-        |> Jason.decode!()
-
-      result =
-        case remote_pack["pack"] do
-          %{"share-files" => true, "can-download" => true, "download-sha256" => sha} ->
-            {:ok,
-             %{
-               sha: sha,
-               url: URI.merge(uri, "/api/pleroma/emoji/packs/#{name}/archive") |> to_string()
-             }}
-
-          %{"fallback-src" => src, "fallback-src-sha256" => sha} when is_binary(src) ->
-            {:ok,
-             %{
-               sha: sha,
-               url: src,
-               fallback: true
-             }}
-
-          _ ->
-            {:error,
-             "The pack was not set as shared and there is no fallback src to download from"}
-        end
-
-      with {:ok, %{sha: sha, url: url} = pinfo} <- result,
-           %{body: archive} <- Tesla.get!(url),
-           {_, true} <- {:checksum, Base.decode16!(sha) == :crypto.hash(:sha256, archive)} do
-        local_name = as || name
-
-        path = Path.join(emoji_path(), local_name)
-
-        pack = %__MODULE__{
-          name: local_name,
-          path: path,
-          files: remote_pack["files"],
-          pack_file: Path.join(path, "pack.json")
-        }
-
-        File.mkdir_p!(pack.path)
-
-        files = Enum.map(remote_pack["files"], fn {_, path} -> to_charlist(path) end)
-        # Fallback cannot contain a pack.json file
-        files = if pinfo[:fallback], do: files, else: ['pack.json' | files]
-
-        {:ok, _} = :zip.unzip(archive, cwd: to_charlist(pack.path), file_list: files)
-
-        # Fallback can't contain a pack.json file, since that would cause the fallback-src-sha256
-        # in it to depend on itself
-        if pinfo[:fallback] do
-          save_pack(pack)
-        end
-
-        :ok
-      end
-    end
-  end
-
-  defp save_pack(pack), do: File.write(pack.pack_file, Jason.encode!(pack, pretty: true))
-
-  @spec save_metadata(map(), t()) :: {:ok, t()} | {:error, File.posix()}
-  def save_metadata(metadata, %__MODULE__{} = pack) do
-    pack = Map.put(pack, :pack, metadata)
-
-    with :ok <- save_pack(pack) do
+  defp save_pack(pack) do
+    with {:ok, json} <- Jason.encode(pack, pretty: true),
+         :ok <- File.write(pack.pack_file, json) do
      {:ok, pack}
    end
  end

-  @spec update_metadata(String.t(), map()) :: {:ok, t()} | {:error, File.posix()}
-  def update_metadata(name, data) do
-    pack = load_pack(name)
-
-    fb_sha_changed? =
-      not is_nil(data["fallback-src"]) and data["fallback-src"] != pack.pack["fallback-src"]
-
-    with {_, true} <- {:update?, fb_sha_changed?},
-         {:ok, %{body: zip}} <- Tesla.get(data["fallback-src"]),
-         {:ok, f_list} <- :zip.unzip(zip, [:memory]),
-         {_, true} <- {:has_all_files?, has_all_files?(pack.files, f_list)} do
-      fallback_sha = :crypto.hash(:sha256, zip) |> Base.encode16()
-
-      data
-      |> Map.put("fallback-src-sha256", fallback_sha)
-      |> save_metadata(pack)
-    else
-      {:update?, _} -> save_metadata(data, pack)
-      e -> e
-    end
-  end
-
-  # Check if all files from the pack.json are in the archive
-  defp has_all_files?(files, f_list) do
-    Enum.all?(files, fn {_, from_manifest} ->
-      List.keyfind(f_list, to_charlist(from_manifest), 0)
-    end)
-  end
-
-  @spec load_pack(String.t()) :: t() | nil
-  def load_pack(name) do
-    pack_file = Path.join([emoji_path(), name, "pack.json"])
-
-    if File.exists?(pack_file) do
-      pack_file
-      |> File.read!()
-      |> from_json()
-      |> Map.put(:pack_file, pack_file)
-      |> Map.put(:path, Path.dirname(pack_file))
-      |> Map.put(:name, name)
-    end
-  end
-
  defp from_json(json) do
    map = Jason.decode!(json)

    struct(__MODULE__, %{files: map["files"], pack: map["pack"]})
  end

-  defp shareable_packs_available?(uri) do
-    uri
-    |> URI.merge("/.well-known/nodeinfo")
-    |> to_string()
-    |> Tesla.get!()
-    |> Map.get(:body)
-    |> Jason.decode!()
-    |> Map.get("links")
-    |> List.last()
-    |> Map.get("href")
-    # Get the actual nodeinfo address and fetch it
-    |> Tesla.get!()
-    |> Map.get(:body)
-    |> Jason.decode!()
-    |> get_in(["metadata", "features"])
-    |> Enum.member?("shareable_emoji_packs")
+  defp validate_shareable_packs_available(uri) do
+    with {:ok, %{"links" => links}} <- uri |> URI.merge("/.well-known/nodeinfo") |> http_get(),
+         # Get the actual nodeinfo address and fetch it
+         {:ok, %{"metadata" => %{"features" => features}}} <-
+           links |> List.last() |> Map.get("href") |> http_get() do
+      if Enum.member?(features, "shareable_emoji_packs") do
+        :ok
+      else
+        {:error, :not_shareable}
+      end
+    end
+  end
+
+  defp validate_not_empty(list) do
+    if Enum.all?(list, fn i -> is_binary(i) and i != "" end) do
+      :ok
+    else
+      {:error, :empty_values}
+    end
+  end
+
+  defp save_file(file, pack, filename) do
+    file_path = Path.join(pack.path, filename)
+    create_subdirs(file_path)
+
+    case file do
+      %Plug.Upload{path: upload_path} ->
+        # Copy the uploaded file from the temporary directory
+        with {:ok, _} <- File.copy(upload_path, file_path), do: :ok
+
+      url when is_binary(url) ->
+        # Download and write the file
+        file_contents = Tesla.get!(url).body
+        File.write(file_path, file_contents)
+    end
+  end
+
+  defp put_emoji(pack, shortcode, filename) do
+    files = Map.put(pack.files, shortcode, filename)
+    %{pack | files: files}
+  end
+
+  defp delete_emoji(pack, shortcode) do
+    files = Map.delete(pack.files, shortcode)
+    %{pack | files: files}
+  end
+
+  defp rename_file(pack, filename, new_filename) do
+    old_path = Path.join(pack.path, filename)
+    new_path = Path.join(pack.path, new_filename)
+    create_subdirs(new_path)
+
+    with :ok <- File.rename(old_path, new_path) do
+      remove_dir_if_empty(old_path, filename)
+    end
+  end
+
+  defp create_subdirs(file_path) do
+    if String.contains?(file_path, "/") do
+      file_path
+      |> Path.dirname()
+      |> File.mkdir_p!()
+    end
+  end
+
+  defp remove_file(pack, shortcode) do
+    with {:ok, filename} <- get_filename(pack, shortcode),
+         emoji <- Path.join(pack.path, filename),
+         :ok <- File.rm(emoji) do
+      remove_dir_if_empty(emoji, filename)
+    end
+  end
+
+  defp remove_dir_if_empty(emoji, filename) do
+    dir = Path.dirname(emoji)
+
+    if String.contains?(filename, "/") and File.ls!(dir) == [] do
+      File.rmdir!(dir)
+    else
+      :ok
+    end
+  end
+
+  defp get_filename(pack, shortcode) do
+    with %{^shortcode => filename} when is_binary(filename) <- pack.files,
+         true <- pack.path |> Path.join(filename) |> File.exists?() do
+      {:ok, filename}
+    else
+      _ -> {:error, :doesnt_exist}
+    end
+  end
+
+  defp http_get(%URI{} = url), do: url |> to_string() |> http_get()
+
+  defp http_get(url) do
+    with {:ok, %{body: body}} <- url |> Pleroma.HTTP.get() do
+      Jason.decode(body)
+    end
+  end
+
+  defp list_packs_dir do
+    emoji_path = emoji_path()
+    # Create the directory first if it does not exist. This is probably the first request made
+    # with the API so it should be sufficient
+    with {:create_dir, :ok} <- {:create_dir, File.mkdir_p(emoji_path)},
+         {:ls, {:ok, results}} <- {:ls, File.ls(emoji_path)} do
+      {:ok, results}
+    else
+      {:create_dir, {:error, e}} -> {:error, :create_dir, e}
+      {:ls, {:error, e}} -> {:error, :ls, e}
+    end
+  end
+
+  defp validate_downloadable(pack) do
+    if downloadable?(pack), do: :ok, else: {:error, :cant_download}
+  end
+
+  defp copy_as(remote_pack, local_name) do
+    path = Path.join(emoji_path(), local_name)
+
+    %__MODULE__{
+      name: local_name,
+      path: path,
+      files: remote_pack["files"],
+      pack_file: Path.join(path, "pack.json")
+    }
+  end
+
+  defp unzip(archive, pack_info, remote_pack, local_pack) do
+    with :ok <- File.mkdir_p!(local_pack.path) do
+      files = Enum.map(remote_pack["files"], fn {_, path} -> to_charlist(path) end)
+      # Fallback cannot contain a pack.json file
+      files = if pack_info[:fallback], do: files, else: ['pack.json' | files]
+
+      :zip.unzip(archive, cwd: to_charlist(local_pack.path), file_list: files)
+    end
+  end
+
+  defp fetch_pack_info(remote_pack, uri, name) do
+    case remote_pack["pack"] do
+      %{"share-files" => true, "can-download" => true, "download-sha256" => sha} ->
+        {:ok,
+         %{
+           sha: sha,
+           url: URI.merge(uri, "/api/pleroma/emoji/packs/#{name}/archive") |> to_string()
+         }}
+
+      %{"fallback-src" => src, "fallback-src-sha256" => sha} when is_binary(src) ->
+        {:ok,
+         %{
+           sha: sha,
+           url: src,
+           fallback: true
+         }}
+
+      _ ->
+        {:error, "The pack was not set as shared and there is no fallback src to download from"}
+    end
+  end
+
+  defp download_archive(url, sha) do
+    with {:ok, %{body: archive}} <- Tesla.get(url) do
+      if Base.decode16!(sha) == :crypto.hash(:sha256, archive) do
+        {:ok, archive}
+      else
+        {:error, :invalid_checksum}
+      end
+    end
+  end
+
+  defp fetch_archive(pack) do
+    hash = :crypto.hash(:md5, File.read!(pack.pack_file))
+
+    case Cachex.get!(:emoji_packs_cache, pack.name) do
+      %{hash: ^hash, pack_data: archive} -> archive
+      _ -> create_archive_and_cache(pack, hash)
+    end
+  end
+
+  defp fallback_sha_changed?(pack, data) do
+    is_binary(data[:"fallback-src"]) and data[:"fallback-src"] != pack.pack["fallback-src"]
+  end
+
+  defp update_sha_and_save_metadata(pack, data) do
+    with {:ok, %{body: zip}} <- Tesla.get(data[:"fallback-src"]),
+         :ok <- validate_has_all_files(pack, zip) do
+      fallback_sha = :sha256 |> :crypto.hash(zip) |> Base.encode16()
+
+      data
+      |> Map.put("fallback-src-sha256", fallback_sha)
+      |> save_metadata(pack)
+    end
+  end
+
+  defp validate_has_all_files(pack, zip) do
+    with {:ok, f_list} <- :zip.unzip(zip, [:memory]) do
+      # Check if all files from the pack.json are in the archive
+      pack.files
+      |> Enum.all?(fn {_, from_manifest} ->
+        List.keyfind(f_list, to_charlist(from_manifest), 0)
+      end)
+      |> if(do: :ok, else: {:error, :incomplete})
+    end
  end
 end
--- a/lib/pleroma/filter.ex
+++ b/lib/pleroma/filter.ex
@ -89,11 +89,10 @@ def delete(%Pleroma.Filter{id: filter_key} = filter) when is_nil(filter_key) do
    |> Repo.delete()
  end

-  def update(%Pleroma.Filter{} = filter) do
-    destination = Map.from_struct(filter)
-
-    Pleroma.Filter.get(filter.filter_id, %{id: filter.user_id})
-    |> cast(destination, [:phrase, :context, :hide, :expires_at, :whole_word])
+  def update(%Pleroma.Filter{} = filter, params) do
+    filter
+    |> cast(params, [:phrase, :context, :hide, :expires_at, :whole_word])
+    |> validate_required([:phrase, :context])
    |> Repo.update()
  end
 end
--- a/lib/pleroma/healthcheck.ex
+++ b/lib/pleroma/healthcheck.ex
@ -29,7 +29,7 @@ defmodule Pleroma.Healthcheck do
  @spec system_info() :: t()
  def system_info do
    %Healthcheck{
-      memory_used: Float.round(:erlang.memory(:total) / 1024 / 1024, 2)
+      memory_used: Float.round(:recon_alloc.memory(:allocated) / 1024 / 1024, 2)
    }
    |> assign_db_info()
    |> assign_job_queue_stats()
--- a/lib/pleroma/helpers/uri_helper.ex
+++ b/lib/pleroma/helpers/uri_helper.ex
@ -24,4 +24,7 @@ def append_param_if_present(%{} = params, param_name, param_value) do
      params
    end
  end
+
+  def maybe_add_base("/" <> uri, base), do: Path.join([base, uri])
+  def maybe_add_base(uri, _base), do: uri
 end
--- a/lib/pleroma/http/adapter_helper/hackney.ex
+++ b/lib/pleroma/http/adapter_helper/hackney.ex
@ -22,22 +22,7 @@ def options(connection_opts \\ [], %URI{} = uri) do
    |> Pleroma.HTTP.AdapterHelper.maybe_add_proxy(proxy)
  end

-  defp add_scheme_opts(opts, %URI{scheme: "http"}), do: opts
-
-  defp add_scheme_opts(opts, %URI{scheme: "https", host: host}) do
-    ssl_opts = [
-      ssl_options: [
-        # Workaround for remote server certificate chain issues
-        partial_chain: &:hackney_connect.partial_chain/1,
-
-        # We don't support TLS v1.3 yet
-        versions: [:tlsv1, :"tlsv1.1", :"tlsv1.2"],
-        server_name_indication: to_charlist(host)
-      ]
-    ]
-
-    Keyword.merge(opts, ssl_opts)
-  end
+  defp add_scheme_opts(opts, _), do: opts

  def after_request(_), do: :ok
 end
--- a/lib/pleroma/maintenance.ex
+++ b/lib/pleroma/maintenance.ex
@ -0,0 +1,37 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Maintenance do
+  alias Pleroma.Repo
+  require Logger
+
+  def vacuum(args) do
+    case args do
+      "analyze" ->
+        Logger.info("Runnning VACUUM ANALYZE.")
+
+        Repo.query!(
+          "vacuum analyze;",
+          [],
+          timeout: :infinity
+        )
+
+      "full" ->
+        Logger.info("Runnning VACUUM FULL.")
+
+        Logger.warn(
+          "Re-packing your entire database may take a while and will consume extra disk space during the process."
+        )
+
+        Repo.query!(
+          "vacuum full;",
+          [],
+          timeout: :infinity
+        )
+
+      _ ->
+        Logger.error("Error: invalid vacuum argument.")
+    end
+  end
+end
--- a/lib/pleroma/marker.ex
+++ b/lib/pleroma/marker.ex
@ -9,24 +9,34 @@ defmodule Pleroma.Marker do
  import Ecto.Query

  alias Ecto.Multi
+  alias Pleroma.Notification
  alias Pleroma.Repo
  alias Pleroma.User
+  alias __MODULE__

  @timelines ["notifications"]
+  @type t :: %__MODULE__{}

  schema "markers" do
    field(:last_read_id, :string, default: "")
    field(:timeline, :string, default: "")
    field(:lock_version, :integer, default: 0)
+    field(:unread_count, :integer, default: 0, virtual: true)

    belongs_to(:user, User, type: FlakeId.Ecto.CompatType)
    timestamps()
  end

+  @doc "Gets markers by user and timeline."
+  @spec get_markers(User.t(), list(String)) :: list(t())
  def get_markers(user, timelines \\ []) do
-    Repo.all(get_query(user, timelines))
+    user
+    |> get_query(timelines)
+    |> unread_count_query()
+    |> Repo.all()
  end

+  @spec upsert(User.t(), map()) :: {:ok | :error, any()}
  def upsert(%User{} = user, attrs) do
    attrs
    |> Map.take(@timelines)
@ -45,6 +55,27 @@ def upsert(%User{} = user, attrs) do
    |> Repo.transaction()
  end

+  @spec multi_set_last_read_id(Multi.t(), User.t(), String.t()) :: Multi.t()
+  def multi_set_last_read_id(multi, %User{} = user, "notifications") do
+    multi
+    |> Multi.run(:counters, fn _repo, _changes ->
+      {:ok, %{last_read_id: Repo.one(Notification.last_read_query(user))}}
+    end)
+    |> Multi.insert(
+      :marker,
+      fn %{counters: attrs} ->
+        %Marker{timeline: "notifications", user_id: user.id}
+        |> struct(attrs)
+        |> Ecto.Changeset.change()
+      end,
+      returning: true,
+      on_conflict: {:replace, [:last_read_id]},
+      conflict_target: [:user_id, :timeline]
+    )
+  end
+
+  def multi_set_last_read_id(multi, _, _), do: multi
+
  defp get_marker(user, timeline) do
    case Repo.find_resource(get_query(user, timeline)) do
      {:ok, marker} -> %__MODULE__{marker | user: user}
@ -71,4 +102,16 @@ defp get_query(user, timelines) do
    |> by_user_id(user.id)
    |> by_timeline(timelines)
  end
+
+  defp unread_count_query(query) do
+    from(
+      q in query,
+      left_join: n in "notifications",
+      on: n.user_id == q.user_id and n.seen == false,
+      group_by: [:id],
+      select_merge: %{
+        unread_count: fragment("count(?)", n.id)
+      }
+    )
+  end
 end
--- a/lib/pleroma/mfa.ex
+++ b/lib/pleroma/mfa.ex
@ -0,0 +1,155 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.MFA do
+  @moduledoc """
+  The MFA context.
+  """
+
+  alias Pleroma.User
+
+  alias Pleroma.MFA.BackupCodes
+  alias Pleroma.MFA.Changeset
+  alias Pleroma.MFA.Settings
+  alias Pleroma.MFA.TOTP
+
+  @doc """
+  Returns MFA methods the user has enabled.
+
+  ## Examples
+
+    iex> Pleroma.MFA.supported_method(User)
+    "totp, u2f"
+  """
+  @spec supported_methods(User.t()) :: String.t()
+  def supported_methods(user) do
+    settings = fetch_settings(user)
+
+    Settings.mfa_methods()
+    |> Enum.reduce([], fn m, acc ->
+      if method_enabled?(m, settings) do
+        acc ++ [m]
+      else
+        acc
+      end
+    end)
+    |> Enum.join(",")
+  end
+
+  @doc "Checks that user enabled MFA"
+  def require?(user) do
+    fetch_settings(user).enabled
+  end
+
+  @doc """
+  Display MFA settings of user
+  """
+  def mfa_settings(user) do
+    settings = fetch_settings(user)
+
+    Settings.mfa_methods()
+    |> Enum.map(fn m -> [m, method_enabled?(m, settings)] end)
+    |> Enum.into(%{enabled: settings.enabled}, fn [a, b] -> {a, b} end)
+  end
+
+  @doc false
+  def fetch_settings(%User{} = user) do
+    user.multi_factor_authentication_settings || %Settings{}
+  end
+
+  @doc "clears backup codes"
+  def invalidate_backup_code(%User{} = user, hash_code) do
+    %{backup_codes: codes} = fetch_settings(user)
+
+    user
+    |> Changeset.cast_backup_codes(codes -- [hash_code])
+    |> User.update_and_set_cache()
+  end
+
+  @doc "generates backup codes"
+  @spec generate_backup_codes(User.t()) :: {:ok, list(binary)} | {:error, String.t()}
+  def generate_backup_codes(%User{} = user) do
+    with codes <- BackupCodes.generate(),
+         hashed_codes <- Enum.map(codes, &Pbkdf2.hash_pwd_salt/1),
+         changeset <- Changeset.cast_backup_codes(user, hashed_codes),
+         {:ok, _} <- User.update_and_set_cache(changeset) do
+      {:ok, codes}
+    else
+      {:error, msg} ->
+        %{error: msg}
+    end
+  end
+
+  @doc """
+  Generates secret key and set delivery_type to 'app' for TOTP method.
+  """
+  @spec setup_totp(User.t()) :: {:ok, User.t()} | {:error, Ecto.Changeset.t()}
+  def setup_totp(user) do
+    user
+    |> Changeset.setup_totp(%{secret: TOTP.generate_secret(), delivery_type: "app"})
+    |> User.update_and_set_cache()
+  end
+
+  @doc """
+  Confirms the TOTP method for user.
+
+  `attrs`:
+    `password` - current user password
+    `code` - TOTP token
+  """
+  @spec confirm_totp(User.t(), map()) :: {:ok, User.t()} | {:error, Ecto.Changeset.t() | atom()}
+  def confirm_totp(%User{} = user, attrs) do
+    with settings <- user.multi_factor_authentication_settings.totp,
+         {:ok, :pass} <- TOTP.validate_token(settings.secret, attrs["code"]) do
+      user
+      |> Changeset.confirm_totp()
+      |> User.update_and_set_cache()
+    end
+  end
+
+  @doc """
+  Disables the TOTP method for user.
+
+  `attrs`:
+    `password` - current user password
+  """
+  @spec disable_totp(User.t()) :: {:ok, User.t()} | {:error, Ecto.Changeset.t()}
+  def disable_totp(%User{} = user) do
+    user
+    |> Changeset.disable_totp()
+    |> Changeset.disable()
+    |> User.update_and_set_cache()
+  end
+
+  @doc """
+  Force disables all MFA methods for user.
+  """
+  @spec disable(User.t()) :: {:ok, User.t()} | {:error, Ecto.Changeset.t()}
+  def disable(%User{} = user) do
+    user
+    |> Changeset.disable_totp()
+    |> Changeset.disable(true)
+    |> User.update_and_set_cache()
+  end
+
+  @doc """
+  Checks if the user has MFA method enabled.
+  """
+  def method_enabled?(method, settings) do
+    with {:ok, %{confirmed: true} = _} <- Map.fetch(settings, method) do
+      true
+    else
+      _ -> false
+    end
+  end
+
+  @doc """
+  Checks if the user has enabled at least one MFA method.
+  """
+  def enabled?(settings) do
+    Settings.mfa_methods()
+    |> Enum.map(fn m -> method_enabled?(m, settings) end)
+    |> Enum.any?()
+  end
+end
--- a/lib/pleroma/mfa/backup_codes.ex
+++ b/lib/pleroma/mfa/backup_codes.ex
@ -0,0 +1,31 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.MFA.BackupCodes do
+  @moduledoc """
+  This module contains functions for generating backup codes.
+  """
+  alias Pleroma.Config
+
+  @config_ns [:instance, :multi_factor_authentication, :backup_codes]
+
+  @doc """
+  Generates backup codes.
+  """
+  @spec generate(Keyword.t()) :: list(String.t())
+  def generate(opts \\ []) do
+    number_of_codes = Keyword.get(opts, :number_of_codes, default_backup_codes_number())
+    code_length = Keyword.get(opts, :length, default_backup_codes_code_length())
+
+    Enum.map(1..number_of_codes, fn _ ->
+      :crypto.strong_rand_bytes(div(code_length, 2))
+      |> Base.encode16(case: :lower)
+    end)
+  end
+
+  defp default_backup_codes_number, do: Config.get(@config_ns ++ [:number], 5)
+
+  defp default_backup_codes_code_length,
+    do: Config.get(@config_ns ++ [:length], 16)
+end
--- a/lib/pleroma/mfa/changeset.ex
+++ b/lib/pleroma/mfa/changeset.ex
@ -0,0 +1,64 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.MFA.Changeset do
+  alias Pleroma.MFA
+  alias Pleroma.MFA.Settings
+  alias Pleroma.User
+
+  def disable(%Ecto.Changeset{} = changeset, force \\ false) do
+    settings =
+      changeset
+      |> Ecto.Changeset.apply_changes()
+      |> MFA.fetch_settings()
+
+    if force || not MFA.enabled?(settings) do
+      put_change(changeset, %Settings{settings | enabled: false})
+    else
+      changeset
+    end
+  end
+
+  def disable_totp(%User{multi_factor_authentication_settings: settings} = user) do
+    user
+    |> put_change(%Settings{settings | totp: %Settings.TOTP{}})
+  end
+
+  def confirm_totp(%User{multi_factor_authentication_settings: settings} = user) do
+    totp_settings = %Settings.TOTP{settings.totp | confirmed: true}
+
+    user
+    |> put_change(%Settings{settings | totp: totp_settings, enabled: true})
+  end
+
+  def setup_totp(%User{} = user, attrs) do
+    mfa_settings = MFA.fetch_settings(user)
+
+    totp_settings =
+      %Settings.TOTP{}
+      |> Ecto.Changeset.cast(attrs, [:secret, :delivery_type])
+
+    user
+    |> put_change(%Settings{mfa_settings | totp: Ecto.Changeset.apply_changes(totp_settings)})
+  end
+
+  def cast_backup_codes(%User{} = user, codes) do
+    user
+    |> put_change(%Settings{
+      user.multi_factor_authentication_settings
+      | backup_codes: codes
+    })
+  end
+
+  defp put_change(%User{} = user, settings) do
+    user
+    |> Ecto.Changeset.change()
+    |> put_change(settings)
+  end
+
+  defp put_change(%Ecto.Changeset{} = changeset, settings) do
+    changeset
+    |> Ecto.Changeset.put_change(:multi_factor_authentication_settings, settings)
+  end
+end
--- a/lib/pleroma/mfa/settings.ex
+++ b/lib/pleroma/mfa/settings.ex
@ -0,0 +1,24 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.MFA.Settings do
+  use Ecto.Schema
+
+  @primary_key false
+
+  @mfa_methods [:totp]
+  embedded_schema do
+    field(:enabled, :boolean, default: false)
+    field(:backup_codes, {:array, :string}, default: [])
+
+    embeds_one :totp, TOTP, on_replace: :delete, primary_key: false do
+      field(:secret, :string)
+      # app | sms
+      field(:delivery_type, :string, default: "app")
+      field(:confirmed, :boolean, default: false)
+    end
+  end
+
+  def mfa_methods, do: @mfa_methods
+end
--- a/lib/pleroma/mfa/token.ex
+++ b/lib/pleroma/mfa/token.ex
@ -0,0 +1,106 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.MFA.Token do
+  use Ecto.Schema
+  import Ecto.Query
+  import Ecto.Changeset
+
+  alias Pleroma.Repo
+  alias Pleroma.User
+  alias Pleroma.Web.OAuth.Authorization
+  alias Pleroma.Web.OAuth.Token, as: OAuthToken
+
+  @expires 300
+
+  schema "mfa_tokens" do
+    field(:token, :string)
+    field(:valid_until, :naive_datetime_usec)
+
+    belongs_to(:user, User, type: FlakeId.Ecto.CompatType)
+    belongs_to(:authorization, Authorization)
+
+    timestamps()
+  end
+
+  def get_by_token(token) do
+    from(
+      t in __MODULE__,
+      where: t.token == ^token,
+      preload: [:user, :authorization]
+    )
+    |> Repo.find_resource()
+  end
+
+  def validate(token) do
+    with {:fetch_token, {:ok, token}} <- {:fetch_token, get_by_token(token)},
+         {:expired, false} <- {:expired, is_expired?(token)} do
+      {:ok, token}
+    else
+      {:expired, _} -> {:error, :expired_token}
+      {:fetch_token, _} -> {:error, :not_found}
+      error -> {:error, error}
+    end
+  end
+
+  def create_token(%User{} = user) do
+    %__MODULE__{}
+    |> change
+    |> assign_user(user)
+    |> put_token
+    |> put_valid_until
+    |> Repo.insert()
+  end
+
+  def create_token(user, authorization) do
+    %__MODULE__{}
+    |> change
+    |> assign_user(user)
+    |> assign_authorization(authorization)
+    |> put_token
+    |> put_valid_until
+    |> Repo.insert()
+  end
+
+  defp assign_user(changeset, user) do
+    changeset
+    |> put_assoc(:user, user)
+    |> validate_required([:user])
+  end
+
+  defp assign_authorization(changeset, authorization) do
+    changeset
+    |> put_assoc(:authorization, authorization)
+    |> validate_required([:authorization])
+  end
+
+  defp put_token(changeset) do
+    changeset
+    |> change(%{token: OAuthToken.Utils.generate_token()})
+    |> validate_required([:token])
+    |> unique_constraint(:token)
+  end
+
+  defp put_valid_until(changeset) do
+    expires_in = NaiveDateTime.add(NaiveDateTime.utc_now(), @expires)
+
+    changeset
+    |> change(%{valid_until: expires_in})
+    |> validate_required([:valid_until])
+  end
+
+  def is_expired?(%__MODULE__{valid_until: valid_until}) do
+    NaiveDateTime.diff(NaiveDateTime.utc_now(), valid_until) > 0
+  end
+
+  def is_expired?(_), do: false
+
+  def delete_expired_tokens do
+    from(
+      q in __MODULE__,
+      where: fragment("?", q.valid_until) < ^Timex.now()
+    )
+    |> Repo.delete_all()
+  end
+end
--- a/lib/pleroma/mfa/totp.ex
+++ b/lib/pleroma/mfa/totp.ex
@ -0,0 +1,86 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.MFA.TOTP do
+  @moduledoc """
+  This module represents functions to create secrets for
+  TOTP Application as well as validate them with a time based token.
+  """
+  alias Pleroma.Config
+
+  @config_ns [:instance, :multi_factor_authentication, :totp]
+
+  @doc """
+  https://github.com/google/google-authenticator/wiki/Key-Uri-Format
+  """
+  def provisioning_uri(secret, label, opts \\ []) do
+    query =
+      %{
+        secret: secret,
+        issuer: Keyword.get(opts, :issuer, default_issuer()),
+        digits: Keyword.get(opts, :digits, default_digits()),
+        period: Keyword.get(opts, :period, default_period())
+      }
+      |> Enum.filter(fn {_, v} -> not is_nil(v) end)
+      |> Enum.into(%{})
+      |> URI.encode_query()
+
+    %URI{scheme: "otpauth", host: "totp", path: "/" <> label, query: query}
+    |> URI.to_string()
+  end
+
+  defp default_period, do: Config.get(@config_ns ++ [:period])
+  defp default_digits, do: Config.get(@config_ns ++ [:digits])
+
+  defp default_issuer,
+    do: Config.get(@config_ns ++ [:issuer], Config.get([:instance, :name]))
+
+  @doc "Creates a random Base 32 encoded string"
+  def generate_secret do
+    Base.encode32(:crypto.strong_rand_bytes(10))
+  end
+
+  @doc "Generates a valid token based on a secret"
+  def generate_token(secret) do
+    :pot.totp(secret)
+  end
+
+  @doc """
+  Validates a given token based on a secret.
+
+  optional parameters:
+  `token_length` default `6`
+  `interval_length` default `30`
+  `window` default 0
+
+  Returns {:ok, :pass} if the token is valid and
+  {:error, :invalid_token} if it is not.
+  """
+  @spec validate_token(String.t(), String.t()) ::
+          {:ok, :pass} | {:error, :invalid_token | :invalid_secret_and_token}
+  def validate_token(secret, token)
+      when is_binary(secret) and is_binary(token) do
+    opts = [
+      token_length: default_digits(),
+      interval_length: default_period()
+    ]
+
+    validate_token(secret, token, opts)
+  end
+
+  def validate_token(_, _), do: {:error, :invalid_secret_and_token}
+
+  @doc "See `validate_token/2`"
+  @spec validate_token(String.t(), String.t(), Keyword.t()) ::
+          {:ok, :pass} | {:error, :invalid_token | :invalid_secret_and_token}
+  def validate_token(secret, token, options)
+      when is_binary(secret) and is_binary(token) do
+    case :pot.valid_totp(token, secret, options) do
+      true -> {:ok, :pass}
+      false -> {:error, :invalid_token}
+    end
+  end
+
+  def validate_token(_, _, _), do: {:error, :invalid_secret_and_token}
+end
--- a/lib/pleroma/notification.ex
+++ b/lib/pleroma/notification.ex
@ -5,8 +5,10 @@
 defmodule Pleroma.Notification do
  use Ecto.Schema

+  alias Ecto.Multi
  alias Pleroma.Activity
  alias Pleroma.FollowingRelationship
+  alias Pleroma.Marker
  alias Pleroma.Notification
  alias Pleroma.Object
  alias Pleroma.Pagination
@ -34,11 +36,30 @@ defmodule Pleroma.Notification do
    timestamps()
  end

+  @spec unread_notifications_count(User.t()) :: integer()
+  def unread_notifications_count(%User{id: user_id}) do
+    from(q in __MODULE__,
+      where: q.user_id == ^user_id and q.seen == false
+    )
+    |> Repo.aggregate(:count, :id)
+  end
+
  def changeset(%Notification{} = notification, attrs) do
    notification
    |> cast(attrs, [:seen])
  end

+  @spec last_read_query(User.t()) :: Ecto.Queryable.t()
+  def last_read_query(user) do
+    from(q in Pleroma.Notification,
+      where: q.user_id == ^user.id,
+      where: q.seen == true,
+      select: type(q.id, :string),
+      limit: 1,
+      order_by: [desc: :id]
+    )
+  end
+
  defp for_user_query_ap_id_opts(user, opts) do
    ap_id_relationships =
      [:block] ++
@ -71,8 +92,9 @@ def for_user_query(user, opts \\ %{}) do
    |> join(:left, [n, a], object in Object,
      on:
        fragment(
-          "(?->>'id') = COALESCE((? -> 'object'::text) ->> 'id'::text)",
+          "(?->>'id') = COALESCE(?->'object'->>'id', ?->>'object')",
          object.data,
+          a.data,
          a.data
        )
    )
@ -185,46 +207,41 @@ def for_user_since(user, date) do
    |> Repo.all()
  end

-  def set_read_up_to(%{id: user_id} = _user, id) do
+  def set_read_up_to(%{id: user_id} = user, id) do
    query =
      from(
        n in Notification,
        where: n.user_id == ^user_id,
        where: n.id <= ^id,
        where: n.seen == false,
-        update: [
-          set: [
-            seen: true,
-            updated_at: ^NaiveDateTime.utc_now()
-          ]
-        ],
        # Ideally we would preload object and activities here
        # but Ecto does not support preloads in update_all
        select: n.id
      )

-    {_, notification_ids} = Repo.update_all(query, [])
+    {:ok, %{ids: {_, notification_ids}}} =
+      Multi.new()
+      |> Multi.update_all(:ids, query, set: [seen: true, updated_at: NaiveDateTime.utc_now()])
+      |> Marker.multi_set_last_read_id(user, "notifications")
+      |> Repo.transaction()

-    Notification
+    for_user_query(user)
    |> where([n], n.id in ^notification_ids)
-    |> join(:inner, [n], activity in assoc(n, :activity))
-    |> join(:left, [n, a], object in Object,
-      on:
-        fragment(
-          "(?->>'id') = COALESCE((? -> 'object'::text) ->> 'id'::text)",
-          object.data,
-          a.data
-        )
-    )
-    |> preload([n, a, o], activity: {a, object: o})
    |> Repo.all()
  end

+  @spec read_one(User.t(), String.t()) ::
+          {:ok, Notification.t()} | {:error, Ecto.Changeset.t()} | nil
  def read_one(%User{} = user, notification_id) do
    with {:ok, %Notification{} = notification} <- get(user, notification_id) do
-      notification
-      |> changeset(%{seen: true})
-      |> Repo.update()
+      Multi.new()
+      |> Multi.update(:update, changeset(notification, %{seen: true}))
+      |> Marker.multi_set_last_read_id(user, "notifications")
+      |> Repo.transaction()
+      |> case do
+        {:ok, %{update: notification}} -> {:ok, notification}
+        {:error, :update, changeset, _} -> {:error, changeset}
+      end
    end
  end

@ -316,8 +333,11 @@ defp do_create_notifications(%Activity{} = activity) do
  # TODO move to sql, too.
  def create_notification(%Activity{} = activity, %User{} = user, do_send \\ true) do
    unless skip?(activity, user) do
-      notification = %Notification{user_id: user.id, activity: activity}
-      {:ok, notification} = Repo.insert(notification)
+      {:ok, %{notification: notification}} =
+        Multi.new()
+        |> Multi.insert(:notification, %Notification{user_id: user.id, activity: activity})
+        |> Marker.multi_set_last_read_id(user, "notifications")
+        |> Repo.transaction()

      if do_send do
        Streamer.stream(["user", "user:notification"], notification)
@ -339,15 +359,10 @@ def get_notified_from_activity(activity, local_only \\ true)

  def get_notified_from_activity(%Activity{data: %{"type" => type}} = activity, local_only)
      when type in ["Create", "Like", "Announce", "Follow", "Move", "EmojiReact"] do
-    potential_receiver_ap_ids =
-      []
-      |> Utils.maybe_notify_to_recipients(activity)
-      |> Utils.maybe_notify_mentioned_recipients(activity)
-      |> Utils.maybe_notify_subscribers(activity)
-      |> Utils.maybe_notify_followers(activity)
-      |> Enum.uniq()
+    potential_receiver_ap_ids = get_potential_receiver_ap_ids(activity)

-    potential_receivers = User.get_users_from_set(potential_receiver_ap_ids, local_only)
+    potential_receivers =
+      User.get_users_from_set(potential_receiver_ap_ids, local_only: local_only)

    notification_enabled_ap_ids =
      potential_receiver_ap_ids
@ -363,6 +378,27 @@ def get_notified_from_activity(%Activity{data: %{"type" => type}} = activity, lo

  def get_notified_from_activity(_, _local_only), do: {[], []}

+  # For some activities, only notify the author of the object
+  def get_potential_receiver_ap_ids(%{data: %{"type" => type, "object" => object_id}})
+      when type in ~w{Like Announce EmojiReact} do
+    case Object.get_cached_by_ap_id(object_id) do
+      %Object{data: %{"actor" => actor}} ->
+        [actor]
+
+      _ ->
+        []
+    end
+  end
+
+  def get_potential_receiver_ap_ids(activity) do
+    []
+    |> Utils.maybe_notify_to_recipients(activity)
+    |> Utils.maybe_notify_mentioned_recipients(activity)
+    |> Utils.maybe_notify_subscribers(activity)
+    |> Utils.maybe_notify_followers(activity)
+    |> Enum.uniq()
+  end
+
  @doc "Filters out AP IDs domain-blocking and not following the activity's actor"
  def exclude_domain_blocker_ap_ids(ap_ids, activity, preloaded_users \\ [])

--- a/lib/pleroma/object.ex
+++ b/lib/pleroma/object.ex
@ -9,11 +9,13 @@ defmodule Pleroma.Object do
  import Ecto.Changeset

  alias Pleroma.Activity
+  alias Pleroma.Config
  alias Pleroma.Object
  alias Pleroma.Object.Fetcher
  alias Pleroma.ObjectTombstone
  alias Pleroma.Repo
  alias Pleroma.User
+  alias Pleroma.Workers.AttachmentsCleanupWorker

  require Logger

@ -138,12 +140,17 @@ def normalize(ap_id, true, options) when is_binary(ap_id) do

  def normalize(_, _, _), do: nil

-  # Owned objects can only be mutated by their owner
-  def authorize_mutation(%Object{data: %{"actor" => actor}}, %User{ap_id: ap_id}),
-    do: actor == ap_id
+  # Owned objects can only be accessed by their owner
+  def authorize_access(%Object{data: %{"actor" => actor}}, %User{ap_id: ap_id}) do
+    if actor == ap_id do
+      :ok
+    else
+      {:error, :forbidden}
+    end
+  end

-  # Legacy objects can be mutated by anybody
-  def authorize_mutation(%Object{}, %User{}), do: true
+  # Legacy objects can be accessed by anybody
+  def authorize_access(%Object{}, %User{}), do: :ok

  @spec get_cached_by_ap_id(String.t()) :: Object.t() | nil
  def get_cached_by_ap_id(ap_id) do
@ -183,27 +190,37 @@ def swap_object_with_tombstone(object) do
  def delete(%Object{data: %{"id" => id}} = object) do
    with {:ok, _obj} = swap_object_with_tombstone(object),
         deleted_activity = Activity.delete_all_by_object_ap_id(id),
-         {:ok, true} <- Cachex.del(:object_cache, "object:#{id}"),
-         {:ok, _} <- Cachex.del(:web_resp_cache, URI.parse(id).path) do
-      with true <- Pleroma.Config.get([:instance, :cleanup_attachments]) do
-        {:ok, _} =
-          Pleroma.Workers.AttachmentsCleanupWorker.enqueue("cleanup_attachments", %{
-            "object" => object
-          })
-      end
+         {:ok, _} <- invalid_object_cache(object) do
+      cleanup_attachments(
+        Config.get([:instance, :cleanup_attachments]),
+        %{"object" => object}
+      )

      {:ok, object, deleted_activity}
    end
  end

-  def prune(%Object{data: %{"id" => id}} = object) do
+  @spec cleanup_attachments(boolean(), %{required(:object) => map()}) ::
+          {:ok, Oban.Job.t() | nil}
+  def cleanup_attachments(true, %{"object" => _} = params) do
+    AttachmentsCleanupWorker.enqueue("cleanup_attachments", params)
+  end
+
+  def cleanup_attachments(_, _), do: {:ok, nil}
+
+  def prune(%Object{data: %{"id" => _id}} = object) do
    with {:ok, object} <- Repo.delete(object),
-         {:ok, true} <- Cachex.del(:object_cache, "object:#{id}"),
-         {:ok, _} <- Cachex.del(:web_resp_cache, URI.parse(id).path) do
+         {:ok, _} <- invalid_object_cache(object) do
      {:ok, object}
    end
  end

+  def invalid_object_cache(%Object{data: %{"id" => id}}) do
+    with {:ok, true} <- Cachex.del(:object_cache, "object:#{id}") do
+      Cachex.del(:web_resp_cache, URI.parse(id).path)
+    end
+  end
+
  def set_cache(%Object{data: %{"id" => ap_id}} = object) do
    Cachex.put(:object_cache, "object:#{ap_id}", object)
    {:ok, object}
--- a/lib/pleroma/plugs/authentication_plug.ex
+++ b/lib/pleroma/plugs/authentication_plug.ex
@ -3,7 +3,6 @@
 # SPDX-License-Identifier: AGPL-3.0-only

 defmodule Pleroma.Plugs.AuthenticationPlug do
-  alias Comeonin.Pbkdf2
  alias Pleroma.Plugs.OAuthScopesPlug
  alias Pleroma.User

@ -17,8 +16,13 @@ def checkpw(password, "$6" <> _ = password_hash) do
    :crypt.crypt(password, password_hash) == password_hash
  end

+  def checkpw(password, "$2" <> _ = password_hash) do
+    # Handle bcrypt passwords for Mastodon migration
+    Bcrypt.verify_pass(password, password_hash)
+  end
+
  def checkpw(password, "$pbkdf2" <> _ = password_hash) do
-    Pbkdf2.checkpw(password, password_hash)
+    Pbkdf2.verify_pass(password, password_hash)
  end

  def checkpw(_password, _password_hash) do
@ -26,6 +30,25 @@ def checkpw(_password, _password_hash) do
    false
  end

+  def maybe_update_password(%User{password_hash: "$2" <> _} = user, password) do
+    do_update_password(user, password)
+  end
+
+  def maybe_update_password(%User{password_hash: "$6" <> _} = user, password) do
+    do_update_password(user, password)
+  end
+
+  def maybe_update_password(user, _), do: {:ok, user}
+
+  defp do_update_password(user, password) do
+    user
+    |> User.password_update_changeset(%{
+      "password" => password,
+      "password_confirmation" => password
+    })
+    |> Pleroma.Repo.update()
+  end
+
  def call(%{assigns: %{user: %User{}}} = conn, _), do: conn

  def call(
@ -37,7 +60,9 @@ def call(
        } = conn,
        _
      ) do
-    if Pbkdf2.checkpw(password, password_hash) do
+    if checkpw(password, password_hash) do
+      {:ok, auth_user} = maybe_update_password(auth_user, password)
+
      conn
      |> assign(:user, auth_user)
      |> OAuthScopesPlug.skip_plug()
@ -47,7 +72,7 @@ def call(
  end

  def call(%{assigns: %{auth_credentials: %{password: _}}} = conn, _) do
-    Pbkdf2.dummy_checkpw()
+    Pbkdf2.no_user_verify()
    conn
  end

--- a/lib/pleroma/plugs/ensure_authenticated_plug.ex
+++ b/lib/pleroma/plugs/ensure_authenticated_plug.ex
@ -15,26 +15,25 @@ def init(options) do
  end

  @impl true
+  def perform(
+        %{
+          assigns: %{
+            auth_credentials: %{password: _},
+            user: %User{multi_factor_authentication_settings: %{enabled: true}}
+          }
+        } = conn,
+        _
+      ) do
+    conn
+    |> render_error(:forbidden, "Two-factor authentication enabled, you must use a access token.")
+    |> halt()
+  end
+
  def perform(%{assigns: %{user: %User{}}} = conn, _) do
    conn
  end

-  def perform(conn, options) do
-    perform =
-      cond do
-        options[:if_func] -> options[:if_func].()
-        options[:unless_func] -> !options[:unless_func].()
-        true -> true
-      end
-
-    if perform do
-      fail(conn)
-    else
-      conn
-    end
-  end
-
-  def fail(conn) do
+  def perform(conn, _) do
    conn
    |> render_error(:forbidden, "Invalid credentials.")
    |> halt()
--- a/lib/pleroma/plugs/federating_plug.ex
+++ b/lib/pleroma/plugs/federating_plug.ex
@ -19,6 +19,9 @@ def call(conn, _opts) do

  def federating?, do: Pleroma.Config.get([:instance, :federating])

+  # Definition for the use in :if_func / :unless_func plug options
+  def federating?(_conn), do: federating?()
+
  defp fail(conn) do
    conn
    |> put_status(404)
--- a/lib/pleroma/plugs/http_security_plug.ex
+++ b/lib/pleroma/plugs/http_security_plug.ex
@ -31,7 +31,7 @@ defp headers do
      {"x-content-type-options", "nosniff"},
      {"referrer-policy", referrer_policy},
      {"x-download-options", "noopen"},
-      {"content-security-policy", csp_string() <> ";"}
+      {"content-security-policy", csp_string()}
    ]

    if report_uri do
@ -43,23 +43,46 @@ defp headers do
        ]
      }

-      headers ++ [{"reply-to", Jason.encode!(report_group)}]
+      [{"reply-to", Jason.encode!(report_group)} | headers]
    else
      headers
    end
  end

+  static_csp_rules = [
+    "default-src 'none'",
+    "base-uri 'self'",
+    "frame-ancestors 'none'",
+    "style-src 'self' 'unsafe-inline'",
+    "font-src 'self'",
+    "manifest-src 'self'"
+  ]
+
+  @csp_start [Enum.join(static_csp_rules, ";") <> ";"]
+
  defp csp_string do
    scheme = Config.get([Pleroma.Web.Endpoint, :url])[:scheme]
    static_url = Pleroma.Web.Endpoint.static_url()
    websocket_url = Pleroma.Web.Endpoint.websocket_url()
    report_uri = Config.get([:http_security, :report_uri])

-    connect_src = "connect-src 'self' #{static_url} #{websocket_url}"
+    img_src = "img-src 'self' data: blob:"
+    media_src = "media-src 'self'"
+
+    {img_src, media_src} =
+      if Config.get([:media_proxy, :enabled]) &&
+           !Config.get([:media_proxy, :proxy_opts, :redirect_on_failure]) do
+        sources = get_proxy_and_attachment_sources()
+        {[img_src, sources], [media_src, sources]}
+      else
+        {[img_src, " https:"], [media_src, " https:"]}
+      end
+
+    connect_src = ["connect-src 'self' blob: ", static_url, ?\s, websocket_url]

    connect_src =
      if Pleroma.Config.get(:env) == :dev do
-        connect_src <> " http://localhost:3035/"
+        [connect_src, " http://localhost:3035/"]
      else
        connect_src
      end
@ -71,27 +94,46 @@ defp csp_string do
        "script-src 'self'"
      end

-    main_part = [
-      "default-src 'none'",
-      "base-uri 'self'",
-      "frame-ancestors 'none'",
-      "img-src 'self' data: blob: https:",
-      "media-src 'self' https:",
-      "style-src 'self' 'unsafe-inline'",
-      "font-src 'self'",
-      "manifest-src 'self'",
-      connect_src,
-      script_src
-    ]
+    report = if report_uri, do: ["report-uri ", report_uri, ";report-to csp-endpoint"]
+    insecure = if scheme == "https", do: "upgrade-insecure-requests"

-    report = if report_uri, do: ["report-uri #{report_uri}; report-to csp-endpoint"], else: []
-
-    insecure = if scheme == "https", do: ["upgrade-insecure-requests"], else: []
-
-    (main_part ++ report ++ insecure)
-    |> Enum.join("; ")
+    @csp_start
+    |> add_csp_param(img_src)
+    |> add_csp_param(media_src)
+    |> add_csp_param(connect_src)
+    |> add_csp_param(script_src)
+    |> add_csp_param(insecure)
+    |> add_csp_param(report)
+    |> :erlang.iolist_to_binary()
  end

+  defp get_proxy_and_attachment_sources do
+    media_proxy_whitelist =
+      Enum.reduce(Config.get([:media_proxy, :whitelist]), [], fn host, acc ->
+        add_source(acc, host)
+      end)
+
+    upload_base_url =
+      if Config.get([Pleroma.Upload, :base_url]),
+        do: URI.parse(Config.get([Pleroma.Upload, :base_url])).host
+
+    s3_endpoint =
+      if Config.get([Pleroma.Upload, :uploader]) == Pleroma.Uploaders.S3,
+        do: URI.parse(Config.get([Pleroma.Uploaders.S3, :public_endpoint])).host
+
+    []
+    |> add_source(upload_base_url)
+    |> add_source(s3_endpoint)
+    |> add_source(media_proxy_whitelist)
+  end
+
+  defp add_source(iodata, nil), do: iodata
+  defp add_source(iodata, source), do: [[?\s, source] | iodata]
+
+  defp add_csp_param(csp_iodata, nil), do: csp_iodata
+
+  defp add_csp_param(csp_iodata, param), do: [[param, ?;] | csp_iodata]
+
  def warn_if_disabled do
    unless Config.get([:http_security, :enabled]) do
      Logger.warn("
--- a/lib/pleroma/scheduled_activity.ex
+++ b/lib/pleroma/scheduled_activity.ex
@ -40,7 +40,7 @@ defp with_media_attachments(
         %{changes: %{params: %{"media_ids" => media_ids} = params}} = changeset
       )
       when is_list(media_ids) do
-    media_attachments = Utils.attachments_from_ids(%{"media_ids" => media_ids})
+    media_attachments = Utils.attachments_from_ids(%{media_ids: media_ids})

    params =
      params
--- a/lib/pleroma/stats.ex
+++ b/lib/pleroma/stats.ex
@ -91,7 +91,7 @@ def calculate_stat_data do
      peers: peers,
      stats: %{
        domain_count: domain_count,
-        status_count: status_count,
+        status_count: status_count || 0,
        user_count: user_count
      }
    }
--- a/lib/pleroma/upload.ex
+++ b/lib/pleroma/upload.ex
@ -134,7 +134,7 @@ defp prepare_upload(%Plug.Upload{} = file, opts) do
    end
  end

-  defp prepare_upload(%{"img" => "data:image/" <> image_data}, opts) do
+  defp prepare_upload(%{img: "data:image/" <> image_data}, opts) do
    parsed = Regex.named_captures(~r/(?<filetype>jpeg|png|gif);base64,(?<data>.*)/, image_data)
    data = Base.decode64!(parsed["data"], ignore: :whitespace)
    hash = String.downcase(Base.encode16(:crypto.hash(:sha256, data)))
--- a/lib/pleroma/user.ex
+++ b/lib/pleroma/user.ex
@ -9,7 +9,6 @@ defmodule Pleroma.User do
  import Ecto.Query
  import Ecto, only: [assoc: 2]

-  alias Comeonin.Pbkdf2
  alias Ecto.Multi
  alias Pleroma.Activity
  alias Pleroma.Config
@ -20,6 +19,7 @@ defmodule Pleroma.User do
  alias Pleroma.Formatter
  alias Pleroma.HTML
  alias Pleroma.Keys
+  alias Pleroma.MFA
  alias Pleroma.Notification
  alias Pleroma.Object
  alias Pleroma.Registration
@ -29,7 +29,9 @@ defmodule Pleroma.User do
  alias Pleroma.UserRelationship
  alias Pleroma.Web
  alias Pleroma.Web.ActivityPub.ActivityPub
+  alias Pleroma.Web.ActivityPub.Builder
  alias Pleroma.Web.ActivityPub.ObjectValidators.Types
+  alias Pleroma.Web.ActivityPub.Pipeline
  alias Pleroma.Web.ActivityPub.Utils
  alias Pleroma.Web.CommonAPI
  alias Pleroma.Web.CommonAPI.Utils, as: CommonUtils
@ -113,7 +115,6 @@ defmodule Pleroma.User do
    field(:is_admin, :boolean, default: false)
    field(:show_role, :boolean, default: true)
    field(:settings, :map, default: nil)
-    field(:magic_key, :string, default: nil)
    field(:uri, Types.Uri, default: nil)
    field(:hide_followers_count, :boolean, default: false)
    field(:hide_follows_count, :boolean, default: false)
@ -189,6 +190,12 @@ defmodule Pleroma.User do
    # `:subscribers` is deprecated (replaced with `subscriber_users` relation)
    field(:subscribers, {:array, :string}, default: [])

+    embeds_one(
+      :multi_factor_authentication_settings,
+      MFA.Settings,
+      on_replace: :delete
+    )
+
    timestamps()
  end

@ -298,8 +305,13 @@ def invisible?(_), do: false

  def avatar_url(user, options \\ []) do
    case user.avatar do
-      %{"url" => [%{"href" => href} | _]} -> href
-      _ -> !options[:no_default] && "#{Web.base_url()}/images/avi.png"
+      %{"url" => [%{"href" => href} | _]} ->
+        href
+
+      _ ->
+        unless options[:no_default] do
+          Config.get([:assets, :default_user_avatar], "#{Web.base_url()}/images/avi.png")
+        end
    end
  end

@ -387,7 +399,6 @@ def remote_user_changeset(struct \\ %User{local: false}, params) do
        :banner,
        :locked,
        :last_refreshed_at,
-        :magic_key,
        :uri,
        :follower_address,
        :following_address,
@ -527,9 +538,10 @@ def update_as_admin_changeset(struct, params) do
    |> delete_change(:also_known_as)
    |> unique_constraint(:email)
    |> validate_format(:email, @email_regex)
+    |> validate_inclusion(:actor_type, ["Person", "Service"])
  end

-  @spec update_as_admin(%User{}, map) :: {:ok, User.t()} | {:error, Ecto.Changeset.t()}
+  @spec update_as_admin(User.t(), map()) :: {:ok, User.t()} | {:error, Changeset.t()}
  def update_as_admin(user, params) do
    params = Map.put(params, "password_confirmation", params["password"])
    changeset = update_as_admin_changeset(user, params)
@ -550,7 +562,7 @@ def password_update_changeset(struct, params) do
    |> put_change(:password_reset_pending, false)
  end

-  @spec reset_password(User.t(), map) :: {:ok, User.t()} | {:error, Ecto.Changeset.t()}
+  @spec reset_password(User.t(), map()) :: {:ok, User.t()} | {:error, Changeset.t()}
  def reset_password(%User{} = user, params) do
    reset_password(user, user, params)
  end
@ -743,7 +755,19 @@ def unfollow(%User{ap_id: ap_id}, %User{ap_id: ap_id}) do
    {:error, "Not subscribed!"}
  end

+  @spec unfollow(User.t(), User.t()) :: {:ok, User.t(), Activity.t()} | {:error, String.t()}
  def unfollow(%User{} = follower, %User{} = followed) do
+    case do_unfollow(follower, followed) do
+      {:ok, follower, followed} ->
+        {:ok, follower, Utils.fetch_latest_follow(follower, followed)}
+
+      error ->
+        error
+    end
+  end
+
+  @spec do_unfollow(User.t(), User.t()) :: {:ok, User.t(), User.t()} | {:error, String.t()}
+  defp do_unfollow(%User{} = follower, %User{} = followed) do
    case get_follow_state(follower, followed) do
      state when state in [:follow_pending, :follow_accept] ->
        FollowingRelationship.unfollow(follower, followed)
@ -754,7 +778,7 @@ def unfollow(%User{} = follower, %User{} = followed) do
          |> update_following_count()
          |> set_cache()

-        {:ok, follower, Utils.fetch_latest_follow(follower, followed)}
+        {:ok, follower, followed}

      nil ->
        {:error, "Not subscribed!"}
@ -927,6 +951,7 @@ def get_cached_by_nickname_or_id(nickname_or_id, opts \\ []) do
    end
  end

+  @spec get_by_nickname(String.t()) :: User.t() | nil
  def get_by_nickname(nickname) do
    Repo.get_by(User, nickname: nickname) ||
      if Regex.match?(~r(@#{Pleroma.Web.Endpoint.host()})i, nickname) do
@ -1184,8 +1209,9 @@ def increment_unread_conversation_count(conversation, %User{local: true} = user)

  def increment_unread_conversation_count(_, user), do: {:ok, user}

-  @spec get_users_from_set([String.t()], boolean()) :: [User.t()]
-  def get_users_from_set(ap_ids, local_only \\ true) do
+  @spec get_users_from_set([String.t()], keyword()) :: [User.t()]
+  def get_users_from_set(ap_ids, opts \\ []) do
+    local_only = Keyword.get(opts, :local_only, true)
    criteria = %{ap_id: ap_ids, deactivated: false}
    criteria = if local_only, do: Map.put(criteria, :local, true), else: criteria

@ -1197,7 +1223,9 @@ def get_users_from_set(ap_ids, local_only \\ true) do
  def get_recipients_from_activity(%Activity{recipients: to, actor: actor}) do
    to = [actor | to]

-    User.Query.build(%{recipients_from_activity: to, local: true, deactivated: false})
+    query = User.Query.build(%{recipients_from_activity: to, local: true, deactivated: false})
+
+    query
    |> Repo.all()
  end

@ -1393,15 +1421,13 @@ def deactivate(%User{} = user, status) do
      user
      |> get_followers()
      |> Enum.filter(& &1.local)
-      |> Enum.each(fn follower ->
-        follower |> update_following_count() |> set_cache()
-      end)
+      |> Enum.each(&set_cache(update_following_count(&1)))

      # Only update local user counts, remote will be update during the next pull.
      user
      |> get_friends()
      |> Enum.filter(& &1.local)
-      |> Enum.each(&update_follower_count/1)
+      |> Enum.each(&do_unfollow(user, &1))

      {:ok, user}
    end
@ -1423,12 +1449,29 @@ def delete(%User{} = user) do
    BackgroundWorker.enqueue("delete_user", %{"user_id" => user.id})
  end

+  defp delete_and_invalidate_cache(%User{} = user) do
+    invalidate_cache(user)
+    Repo.delete(user)
+  end
+
+  defp delete_or_deactivate(%User{local: false} = user), do: delete_and_invalidate_cache(user)
+
+  defp delete_or_deactivate(%User{local: true} = user) do
+    status = account_status(user)
+
+    if status == :confirmation_pending do
+      delete_and_invalidate_cache(user)
+    else
+      user
+      |> change(%{deactivated: true, email: nil})
+      |> update_and_set_cache()
+    end
+  end
+
  def perform(:force_password_reset, user), do: force_password_reset(user)

  @spec perform(atom(), User.t()) :: {:ok, User.t()}
  def perform(:delete, %User{} = user) do
-    {:ok, _user} = ActivityPub.delete(user)
-
    # Remove all relationships
    user
    |> get_followers()
@ -1446,14 +1489,7 @@ def perform(:delete, %User{} = user) do

    delete_user_activities(user)

-    if user.local do
-      user
-      |> change(%{deactivated: true, email: nil})
-      |> update_and_set_cache()
-    else
-      invalidate_cache(user)
-      Repo.delete(user)
-    end
+    delete_or_deactivate(user)
  end

  def perform(:deactivate_async, user, status), do: deactivate(user, status)
@ -1538,37 +1574,42 @@ def follow_import(%User{} = follower, followed_identifiers)
    })
  end

-  def delete_user_activities(%User{ap_id: ap_id}) do
+  def delete_user_activities(%User{ap_id: ap_id} = user) do
    ap_id
    |> Activity.Queries.by_actor()
    |> RepoStreamer.chunk_stream(50)
-    |> Stream.each(fn activities -> Enum.each(activities, &delete_activity/1) end)
+    |> Stream.each(fn activities ->
+      Enum.each(activities, fn activity -> delete_activity(activity, user) end)
+    end)
    |> Stream.run()
  end

-  defp delete_activity(%{data: %{"type" => "Create"}} = activity) do
-    activity
-    |> Object.normalize()
-    |> ActivityPub.delete()
+  defp delete_activity(%{data: %{"type" => "Create", "object" => object}} = activity, user) do
+    with {_, %Object{}} <- {:find_object, Object.get_by_ap_id(object)},
+         {:ok, delete_data, _} <- Builder.delete(user, object) do
+      Pipeline.common_pipeline(delete_data, local: user.local)
+    else
+      {:find_object, nil} ->
+        # We have the create activity, but not the object, it was probably pruned.
+        # Insert a tombstone and try again
+        with {:ok, tombstone_data, _} <- Builder.tombstone(user.ap_id, object),
+             {:ok, _tombstone} <- Object.create(tombstone_data) do
+          delete_activity(activity, user)
+        end
+
+      e ->
+        Logger.error("Could not delete #{object} created by #{activity.data["ap_id"]}")
+        Logger.error("Error: #{inspect(e)}")
+    end
  end

-  defp delete_activity(%{data: %{"type" => "Like"}} = activity) do
-    object = Object.normalize(activity)
-
-    activity.actor
-    |> get_cached_by_ap_id()
-    |> ActivityPub.unlike(object)
+  defp delete_activity(%{data: %{"type" => type}} = activity, user)
+       when type in ["Like", "Announce"] do
+    {:ok, undo, _} = Builder.undo(user, activity)
+    Pipeline.common_pipeline(undo, local: user.local)
  end

-  defp delete_activity(%{data: %{"type" => "Announce"}} = activity) do
-    object = Object.normalize(activity)
-
-    activity.actor
-    |> get_cached_by_ap_id()
-    |> ActivityPub.unannounce(object)
-  end
-
-  defp delete_activity(_activity), do: "Doing nothing"
+  defp delete_activity(_activity, _user), do: "Doing nothing"

  def html_filter_policy(%User{no_rich_text: true}) do
    Pleroma.HTML.Scrubber.TwitterText
@ -1579,12 +1620,19 @@ def html_filter_policy(_), do: Pleroma.Config.get([:markup, :scrub_policy])
  def fetch_by_ap_id(ap_id), do: ActivityPub.make_user_from_ap_id(ap_id)

  def get_or_fetch_by_ap_id(ap_id) do
-    user = get_cached_by_ap_id(ap_id)
+    cached_user = get_cached_by_ap_id(ap_id)

-    if !is_nil(user) and !needs_update?(user) do
-      {:ok, user}
-    else
-      fetch_by_ap_id(ap_id)
+    maybe_fetched_user = needs_update?(cached_user) && fetch_by_ap_id(ap_id)
+
+    case {cached_user, maybe_fetched_user} do
+      {_, {:ok, %User{} = user}} ->
+        {:ok, user}
+
+      {%User{} = user, _} ->
+        {:ok, user}
+
+      _ ->
+        {:error, :not_found}
    end
  end

@ -1915,7 +1963,7 @@ def get_ap_ids_by_nicknames(nicknames) do
  defp put_password_hash(
         %Ecto.Changeset{valid?: true, changes: %{password: password}} = changeset
       ) do
-    change(changeset, password_hash: Pbkdf2.hashpwsalt(password))
+    change(changeset, password_hash: Pbkdf2.hash_pwd_salt(password))
  end

  defp put_password_hash(changeset), do: changeset
--- a/lib/pleroma/user/query.ex
+++ b/lib/pleroma/user/query.ex
@ -45,6 +45,7 @@ defmodule Pleroma.User.Query do
            is_admin: boolean(),
            is_moderator: boolean(),
            super_users: boolean(),
+            exclude_service_users: boolean(),
            followers: User.t(),
            friends: User.t(),
            recipients_from_activity: [String.t()],
@ -88,6 +89,10 @@ defp compose_query({key, value}, query)
    where(query, [u], ilike(field(u, ^key), ^"%#{value}%"))
  end

+  defp compose_query({:exclude_service_users, _}, query) do
+    where(query, [u], not like(u.ap_id, "%/relay") and not like(u.ap_id, "%/internal/fetch"))
+  end
+
  defp compose_query({key, value}, query)
       when key in @equal_criteria and not_empty_string(value) do
    where(query, [u], ^[{key, value}])
@ -98,7 +103,7 @@ defp compose_query({key, values}, query) when key in @contains_criteria and is_l
  end

  defp compose_query({:tags, tags}, query) when is_list(tags) and length(tags) > 0 do
-    Enum.reduce(tags, query, &prepare_tag_criteria/2)
+    where(query, [u], fragment("? && ?", u.tags, ^tags))
  end

  defp compose_query({:is_admin, _}, query) do
@ -162,20 +167,18 @@ defp compose_query({:friends, %User{id: id}}, query) do
  end

  defp compose_query({:recipients_from_activity, to}, query) do
-    query
-    |> join(:left, [u], r in FollowingRelationship,
-      as: :relationships,
-      on: r.follower_id == u.id
+    following_query =
+      from(u in User,
+        join: f in FollowingRelationship,
+        on: u.id == f.following_id,
+        where: f.state == ^:follow_accept,
+        where: u.follower_address in ^to,
+        select: f.follower_id
+      )
+
+    from(u in query,
+      where: u.ap_id in ^to or u.id in subquery(following_query)
    )
-    |> join(:left, [relationships: r], f in User,
-      as: :following,
-      on: f.id == r.following_id
-    )
-    |> where(
-      [u, following: f, relationships: r],
-      u.ap_id in ^to or (f.follower_address in ^to and r.state == ^:follow_accept)
-    )
-    |> distinct(true)
  end

  defp compose_query({:order_by, key}, query) do
@ -192,10 +195,6 @@ defp compose_query({:limit, limit}, query) do

  defp compose_query(_unsupported_param, query), do: query

-  defp prepare_tag_criteria(tag, query) do
-    or_where(query, [u], fragment("? = any(?)", ^tag, u.tags))
-  end
-
  defp location_query(query, local) do
    where(query, [u], u.local == ^local)
    |> where([u], not is_nil(u.nickname))
--- a/lib/pleroma/user/welcome_message.ex
+++ b/lib/pleroma/user/welcome_message.ex
@ -10,8 +10,8 @@ def post_welcome_message_to_user(user) do
    with %User{} = sender_user <- welcome_user(),
         message when is_binary(message) <- welcome_message() do
      CommonAPI.post(sender_user, %{
-        "visibility" => "direct",
-        "status" => "@#{user.nickname}\n#{message}"
+        visibility: "direct",
+        status: "@#{user.nickname}\n#{message}"
      })
    else
      _ -> {:ok, nil}
--- a/lib/pleroma/user_relationship.ex
+++ b/lib/pleroma/user_relationship.ex
@ -87,6 +87,22 @@ def dictionary(
        source_to_target_rel_types \\ nil,
        target_to_source_rel_types \\ nil
      )
+
+  def dictionary(
+        _source_users,
+        _target_users,
+        [] = _source_to_target_rel_types,
+        [] = _target_to_source_rel_types
+      ) do
+    []
+  end
+
+  def dictionary(
+        source_users,
+        target_users,
+        source_to_target_rel_types,
+        target_to_source_rel_types
+      )
      when is_list(source_users) and is_list(target_users) do
    source_user_ids = User.binary_id(source_users)
    target_user_ids = User.binary_id(target_users)
@ -138,11 +154,16 @@ def view_relationships_option(nil = _reading_user, _actors, _opts) do

  def view_relationships_option(%User{} = reading_user, actors, opts) do
    {source_to_target_rel_types, target_to_source_rel_types} =
-      if opts[:source_mutes_only] do
-        # This option is used for rendering statuses (FE needs `muted` flag for each one anyways)
-        {[:mute], []}
-      else
-        {[:block, :mute, :notification_mute, :reblog_mute], [:block, :inverse_subscription]}
+      case opts[:subset] do
+        :source_mutes ->
+          # Used for statuses rendering (FE needs `muted` flag for each status when statuses load)
+          {[:mute], []}
+
+        nil ->
+          {[:block, :mute, :notification_mute, :reblog_mute], [:block, :inverse_subscription]}
+
+        unknown ->
+          raise "Unsupported :subset option value: #{inspect(unknown)}"
      end

    user_relationships =
@ -153,7 +174,17 @@ def view_relationships_option(%User{} = reading_user, actors, opts) do
        target_to_source_rel_types
      )

-    following_relationships = FollowingRelationship.all_between_user_sets([reading_user], actors)
+    following_relationships =
+      case opts[:subset] do
+        :source_mutes ->
+          []
+
+        nil ->
+          FollowingRelationship.all_between_user_sets([reading_user], actors)
+
+        unknown ->
+          raise "Unsupported :subset option value: #{inspect(unknown)}"
+      end

    %{user_relationships: user_relationships, following_relationships: following_relationships}
  end
--- a/lib/pleroma/web/activity_pub/activity_pub.ex
+++ b/lib/pleroma/web/activity_pub/activity_pub.ex
@ -173,12 +173,6 @@ def insert(map, local \\ true, fake \\ false, bypass_actor_check \\ false) when

      BackgroundWorker.enqueue("fetch_data_for_activity", %{"activity_id" => activity.id})

-      Notification.create_notifications(activity)
-
-      conversation = create_or_bump_conversation(activity, map["actor"])
-      participations = get_participations(conversation)
-      stream_out(activity)
-      stream_out_participations(participations)
      {:ok, activity}
    else
      %Activity{} = activity ->
@ -201,6 +195,15 @@ def insert(map, local \\ true, fake \\ false, bypass_actor_check \\ false) when
    end
  end

+  def notify_and_stream(activity) do
+    Notification.create_notifications(activity)
+
+    conversation = create_or_bump_conversation(activity, activity.actor)
+    participations = get_participations(conversation)
+    stream_out(activity)
+    stream_out_participations(participations)
+  end
+
  defp maybe_create_activity_expiration({:ok, %{data: %{"expires_at" => expires_at}} = activity}) do
    with {:ok, _} <- ActivityExpiration.create(activity, expires_at) do
      {:ok, activity}
@ -285,6 +288,7 @@ defp do_create(%{to: to, actor: actor, context: context, object: object} = param
         _ <- increase_poll_votes_if_vote(create_data),
         {:quick_insert, false, activity} <- {:quick_insert, quick_insert?, activity},
         {:ok, _actor} <- increase_note_count_if_public(actor, activity),
+         _ <- notify_and_stream(activity),
         :ok <- maybe_federate(activity) do
      {:ok, activity}
    else
@ -312,6 +316,7 @@ def listen(%{to: to, actor: actor, context: context, object: object} = params) d
             additional
           ),
         {:ok, activity} <- insert(listen_data, local),
+         _ <- notify_and_stream(activity),
         :ok <- maybe_federate(activity) do
      {:ok, activity}
    end
@ -336,6 +341,7 @@ def accept_or_reject(type, %{to: to, actor: actor, object: object} = params) do
           %{"to" => to, "type" => type, "actor" => actor.ap_id, "object" => object}
           |> Utils.maybe_put("id", activity_id),
         {:ok, activity} <- insert(data, local),
+         _ <- notify_and_stream(activity),
         :ok <- maybe_federate(activity) do
      {:ok, activity}
    end
@ -355,140 +361,12 @@ def update(%{to: to, cc: cc, actor: actor, object: object} = params) do
         },
         data <- Utils.maybe_put(data, "id", activity_id),
         {:ok, activity} <- insert(data, local),
+         _ <- notify_and_stream(activity),
         :ok <- maybe_federate(activity) do
      {:ok, activity}
    end
  end

-  @spec react_with_emoji(User.t(), Object.t(), String.t(), keyword()) ::
-          {:ok, Activity.t(), Object.t()} | {:error, any()}
-  def react_with_emoji(user, object, emoji, options \\ []) do
-    with {:ok, result} <-
-           Repo.transaction(fn -> do_react_with_emoji(user, object, emoji, options) end) do
-      result
-    end
-  end
-
-  defp do_react_with_emoji(user, object, emoji, options) do
-    with local <- Keyword.get(options, :local, true),
-         activity_id <- Keyword.get(options, :activity_id, nil),
-         true <- Pleroma.Emoji.is_unicode_emoji?(emoji),
-         reaction_data <- make_emoji_reaction_data(user, object, emoji, activity_id),
-         {:ok, activity} <- insert(reaction_data, local),
-         {:ok, object} <- add_emoji_reaction_to_object(activity, object),
-         :ok <- maybe_federate(activity) do
-      {:ok, activity, object}
-    else
-      false -> {:error, false}
-      {:error, error} -> Repo.rollback(error)
-    end
-  end
-
-  @spec unreact_with_emoji(User.t(), String.t(), keyword()) ::
-          {:ok, Activity.t(), Object.t()} | {:error, any()}
-  def unreact_with_emoji(user, reaction_id, options \\ []) do
-    with {:ok, result} <-
-           Repo.transaction(fn -> do_unreact_with_emoji(user, reaction_id, options) end) do
-      result
-    end
-  end
-
-  defp do_unreact_with_emoji(user, reaction_id, options) do
-    with local <- Keyword.get(options, :local, true),
-         activity_id <- Keyword.get(options, :activity_id, nil),
-         user_ap_id <- user.ap_id,
-         %Activity{actor: ^user_ap_id} = reaction_activity <- Activity.get_by_ap_id(reaction_id),
-         object <- Object.normalize(reaction_activity),
-         unreact_data <- make_undo_data(user, reaction_activity, activity_id),
-         {:ok, activity} <- insert(unreact_data, local),
-         {:ok, object} <- remove_emoji_reaction_from_object(reaction_activity, object),
-         :ok <- maybe_federate(activity) do
-      {:ok, activity, object}
-    else
-      {:error, error} -> Repo.rollback(error)
-    end
-  end
-
-  @spec unlike(User.t(), Object.t(), String.t() | nil, boolean()) ::
-          {:ok, Activity.t(), Activity.t(), Object.t()} | {:ok, Object.t()} | {:error, any()}
-  def unlike(%User{} = actor, %Object{} = object, activity_id \\ nil, local \\ true) do
-    with {:ok, result} <-
-           Repo.transaction(fn -> do_unlike(actor, object, activity_id, local) end) do
-      result
-    end
-  end
-
-  defp do_unlike(actor, object, activity_id, local) do
-    with %Activity{} = like_activity <- get_existing_like(actor.ap_id, object),
-         unlike_data <- make_unlike_data(actor, like_activity, activity_id),
-         {:ok, unlike_activity} <- insert(unlike_data, local),
-         {:ok, _activity} <- Repo.delete(like_activity),
-         {:ok, object} <- remove_like_from_object(like_activity, object),
-         :ok <- maybe_federate(unlike_activity) do
-      {:ok, unlike_activity, like_activity, object}
-    else
-      nil -> {:ok, object}
-      {:error, error} -> Repo.rollback(error)
-    end
-  end
-
-  @spec announce(User.t(), Object.t(), String.t() | nil, boolean(), boolean()) ::
-          {:ok, Activity.t(), Object.t()} | {:error, any()}
-  def announce(
-        %User{ap_id: _} = user,
-        %Object{data: %{"id" => _}} = object,
-        activity_id \\ nil,
-        local \\ true,
-        public \\ true
-      ) do
-    with {:ok, result} <-
-           Repo.transaction(fn -> do_announce(user, object, activity_id, local, public) end) do
-      result
-    end
-  end
-
-  defp do_announce(user, object, activity_id, local, public) do
-    with true <- is_announceable?(object, user, public),
-         object <- Object.get_by_id(object.id),
-         announce_data <- make_announce_data(user, object, activity_id, public),
-         {:ok, activity} <- insert(announce_data, local),
-         {:ok, object} <- add_announce_to_object(activity, object),
-         :ok <- maybe_federate(activity) do
-      {:ok, activity, object}
-    else
-      false -> {:error, false}
-      {:error, error} -> Repo.rollback(error)
-    end
-  end
-
-  @spec unannounce(User.t(), Object.t(), String.t() | nil, boolean()) ::
-          {:ok, Activity.t(), Object.t()} | {:ok, Object.t()} | {:error, any()}
-  def unannounce(
-        %User{} = actor,
-        %Object{} = object,
-        activity_id \\ nil,
-        local \\ true
-      ) do
-    with {:ok, result} <-
-           Repo.transaction(fn -> do_unannounce(actor, object, activity_id, local) end) do
-      result
-    end
-  end
-
-  defp do_unannounce(actor, object, activity_id, local) do
-    with %Activity{} = announce_activity <- get_existing_announce(actor.ap_id, object),
-         unannounce_data <- make_unannounce_data(actor, announce_activity, activity_id),
-         {:ok, unannounce_activity} <- insert(unannounce_data, local),
-         :ok <- maybe_federate(unannounce_activity),
-         {:ok, _activity} <- Repo.delete(announce_activity),
-         {:ok, object} <- remove_announce_from_object(announce_activity, object) do
-      {:ok, unannounce_activity, object}
-    else
-      nil -> {:ok, object}
-      {:error, error} -> Repo.rollback(error)
-    end
-  end
-
  @spec follow(User.t(), User.t(), String.t() | nil, boolean()) ::
          {:ok, Activity.t()} | {:error, any()}
  def follow(follower, followed, activity_id \\ nil, local \\ true) do
@ -501,6 +379,7 @@ def follow(follower, followed, activity_id \\ nil, local \\ true) do
  defp do_follow(follower, followed, activity_id, local) do
    with data <- make_follow_data(follower, followed, activity_id),
         {:ok, activity} <- insert(data, local),
+         _ <- notify_and_stream(activity),
         :ok <- maybe_federate(activity) do
      {:ok, activity}
    else
@ -522,6 +401,7 @@ defp do_unfollow(follower, followed, activity_id, local) do
         {:ok, follow_activity} <- update_follow_state(follow_activity, "cancelled"),
         unfollow_data <- make_unfollow_data(follower, followed, follow_activity, activity_id),
         {:ok, activity} <- insert(unfollow_data, local),
+         _ <- notify_and_stream(activity),
         :ok <- maybe_federate(activity) do
      {:ok, activity}
    else
@ -530,67 +410,6 @@ defp do_unfollow(follower, followed, activity_id, local) do
    end
  end

-  @spec delete(User.t() | Object.t(), keyword()) :: {:ok, User.t() | Object.t()} | {:error, any()}
-  def delete(entity, options \\ []) do
-    with {:ok, result} <- Repo.transaction(fn -> do_delete(entity, options) end) do
-      result
-    end
-  end
-
-  defp do_delete(%User{ap_id: ap_id, follower_address: follower_address} = user, _) do
-    with data <- %{
-           "to" => [follower_address],
-           "type" => "Delete",
-           "actor" => ap_id,
-           "object" => %{"type" => "Person", "id" => ap_id}
-         },
-         {:ok, activity} <- insert(data, true, true, true),
-         :ok <- maybe_federate(activity) do
-      {:ok, user}
-    end
-  end
-
-  defp do_delete(%Object{data: %{"id" => id, "actor" => actor}} = object, options) do
-    local = Keyword.get(options, :local, true)
-    activity_id = Keyword.get(options, :activity_id, nil)
-    actor = Keyword.get(options, :actor, actor)
-
-    user = User.get_cached_by_ap_id(actor)
-    to = (object.data["to"] || []) ++ (object.data["cc"] || [])
-
-    with create_activity <- Activity.get_create_by_object_ap_id(id),
-         data <-
-           %{
-             "type" => "Delete",
-             "actor" => actor,
-             "object" => id,
-             "to" => to,
-             "deleted_activity_id" => create_activity && create_activity.id
-           }
-           |> maybe_put("id", activity_id),
-         {:ok, activity} <- insert(data, local, false),
-         {:ok, object, _create_activity} <- Object.delete(object),
-         stream_out_participations(object, user),
-         _ <- decrease_replies_count_if_reply(object),
-         {:ok, _actor} <- decrease_note_count_if_public(user, object),
-         :ok <- maybe_federate(activity) do
-      {:ok, activity}
-    else
-      {:error, error} ->
-        Repo.rollback(error)
-    end
-  end
-
-  defp do_delete(%Object{data: %{"type" => "Tombstone", "id" => ap_id}}, _) do
-    activity =
-      ap_id
-      |> Activity.Queries.by_object_id()
-      |> Activity.Queries.by_type("Delete")
-      |> Repo.one()
-
-    {:ok, activity}
-  end
-
  @spec block(User.t(), User.t(), String.t() | nil, boolean()) ::
          {:ok, Activity.t()} | {:error, any()}
  def block(blocker, blocked, activity_id \\ nil, local \\ true) do
@ -601,7 +420,6 @@ def block(blocker, blocked, activity_id \\ nil, local \\ true) do
  end

  defp do_block(blocker, blocked, activity_id, local) do
-    outgoing_blocks = Config.get([:activitypub, :outgoing_blocks])
    unfollow_blocked = Config.get([:activitypub, :unfollow_blocked])

    if unfollow_blocked do
@ -609,9 +427,9 @@ defp do_block(blocker, blocked, activity_id, local) do
      if follow_activity, do: unfollow(blocker, blocked, nil, local)
    end

-    with true <- outgoing_blocks,
-         block_data <- make_block_data(blocker, blocked, activity_id),
+    with block_data <- make_block_data(blocker, blocked, activity_id),
         {:ok, activity} <- insert(block_data, local),
+         _ <- notify_and_stream(activity),
         :ok <- maybe_federate(activity) do
      {:ok, activity}
    else
@ -619,27 +437,6 @@ defp do_block(blocker, blocked, activity_id, local) do
    end
  end

-  @spec unblock(User.t(), User.t(), String.t() | nil, boolean()) ::
-          {:ok, Activity.t()} | {:error, any()} | nil
-  def unblock(blocker, blocked, activity_id \\ nil, local \\ true) do
-    with {:ok, result} <-
-           Repo.transaction(fn -> do_unblock(blocker, blocked, activity_id, local) end) do
-      result
-    end
-  end
-
-  defp do_unblock(blocker, blocked, activity_id, local) do
-    with %Activity{} = block_activity <- fetch_latest_block(blocker, blocked),
-         unblock_data <- make_unblock_data(blocker, blocked, block_activity, activity_id),
-         {:ok, activity} <- insert(unblock_data, local),
-         :ok <- maybe_federate(activity) do
-      {:ok, activity}
-    else
-      nil -> nil
-      {:error, error} -> Repo.rollback(error)
-    end
-  end
-
  @spec flag(map()) :: {:ok, Activity.t()} | {:error, any()}
  def flag(
        %{
@ -666,6 +463,7 @@ def flag(
    with flag_data <- make_flag_data(params, additional),
         {:ok, activity} <- insert(flag_data, local),
         {:ok, stripped_activity} <- strip_report_status_data(activity),
+         _ <- notify_and_stream(activity),
         :ok <- maybe_federate(stripped_activity) do
      User.all_superusers()
      |> Enum.filter(fn user -> not is_nil(user.email) end)
@ -689,7 +487,8 @@ def move(%User{} = origin, %User{} = target, local \\ true) do
    }

    with true <- origin.ap_id in target.also_known_as,
-         {:ok, activity} <- insert(params, local) do
+         {:ok, activity} <- insert(params, local),
+         _ <- notify_and_stream(activity) do
      maybe_federate(activity)

      BackgroundWorker.enqueue("move_following", %{
@ -750,14 +549,27 @@ def fetch_latest_activity_id_for_context(context, opts \\ %{}) do
    |> Repo.one()
  end

-  @spec fetch_public_activities(map(), Pagination.type()) :: [Activity.t()]
-  def fetch_public_activities(opts \\ %{}, pagination \\ :keyset) do
+  @spec fetch_public_or_unlisted_activities(map(), Pagination.type()) :: [Activity.t()]
+  def fetch_public_or_unlisted_activities(opts \\ %{}, pagination \\ :keyset) do
    opts = Map.drop(opts, ["user"])

-    [Constants.as_public()]
-    |> fetch_activities_query(opts)
-    |> restrict_unlisted()
-    |> Pagination.fetch_paginated(opts, pagination)
+    query = fetch_activities_query([Constants.as_public()], opts)
+
+    query =
+      if opts["restrict_unlisted"] do
+        restrict_unlisted(query)
+      else
+        query
+      end
+
+    Pagination.fetch_paginated(query, opts, pagination)
+  end
+
+  @spec fetch_public_activities(map(), Pagination.type()) :: [Activity.t()]
+  def fetch_public_activities(opts \\ %{}, pagination \\ :keyset) do
+    opts
+    |> Map.put("restrict_unlisted", true)
+    |> fetch_public_or_unlisted_activities(pagination)
  end

  @valid_visibilities ~w[direct unlisted public private]
@ -1357,7 +1169,7 @@ def fetch_favourites(user, params \\ %{}, pagination \\ :keyset) do
    |> Activity.with_joined_object()
    |> Object.with_joined_activity()
    |> select([_like, object, activity], %{activity | object: object})
-    |> order_by([like, _, _], desc: like.id)
+    |> order_by([like, _, _], desc_nulls_last: like.id)
    |> Pagination.fetch_paginated(
      Map.merge(params, %{"skip_order" => true}),
      pagination,
--- a/lib/pleroma/web/activity_pub/activity_pub_controller.ex
+++ b/lib/pleroma/web/activity_pub/activity_pub_controller.ex
@ -21,6 +21,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
  alias Pleroma.Web.ActivityPub.UserView
  alias Pleroma.Web.ActivityPub.Utils
  alias Pleroma.Web.ActivityPub.Visibility
+  alias Pleroma.Web.Endpoint
  alias Pleroma.Web.FederatingPlug
  alias Pleroma.Web.Federator

@ -34,7 +35,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do

  plug(
    EnsureAuthenticatedPlug,
-    [unless_func: &FederatingPlug.federating?/0] when action not in @federating_only_actions
+    [unless_func: &FederatingPlug.federating?/1] when action not in @federating_only_actions
  )

  # Note: :following and :followers must be served even without authentication (as via :api)
@ -75,8 +76,8 @@ def user(conn, %{"nickname" => nickname}) do
    end
  end

-  def object(conn, %{"uuid" => uuid}) do
-    with ap_id <- o_status_url(conn, :object, uuid),
+  def object(conn, _) do
+    with ap_id <- Endpoint.url() <> conn.request_path,
         %Object{} = object <- Object.get_cached_by_ap_id(ap_id),
         {_, true} <- {:public?, Visibility.is_public?(object)} do
      conn
@ -101,8 +102,8 @@ def track_object_fetch(conn, object_id) do
    conn
  end

-  def activity(conn, %{"uuid" => uuid}) do
-    with ap_id <- o_status_url(conn, :activity, uuid),
+  def activity(conn, _params) do
+    with ap_id <- Endpoint.url() <> conn.request_path,
         %Activity{} = activity <- Activity.normalize(ap_id),
         {_, true} <- {:public?, Visibility.is_public?(activity)} do
      conn
@ -396,7 +397,10 @@ def read_inbox(%{assigns: %{user: %User{nickname: as_nickname}}} = conn, %{
    |> json(err)
  end

-  defp handle_user_activity(%User{} = user, %{"type" => "Create"} = params) do
+  defp handle_user_activity(
+         %User{} = user,
+         %{"type" => "Create", "object" => %{"type" => "Note"}} = params
+       ) do
    object =
      params["object"]
      |> Map.merge(Map.take(params, ["to", "cc"]))
@ -415,7 +419,8 @@ defp handle_user_activity(%User{} = user, %{"type" => "Create"} = params) do
  defp handle_user_activity(%User{} = user, %{"type" => "Delete"} = params) do
    with %Object{} = object <- Object.normalize(params["object"]),
         true <- user.is_moderator || user.ap_id == object.data["actor"],
-         {:ok, delete} <- ActivityPub.delete(object) do
+         {:ok, delete_data, _} <- Builder.delete(user, object.data["id"]),
+         {:ok, delete, _} <- Pipeline.common_pipeline(delete_data, local: true) do
      {:ok, delete}
    else
      _ -> {:error, dgettext("errors", "Can't delete object")}
--- a/lib/pleroma/web/activity_pub/builder.ex
+++ b/lib/pleroma/web/activity_pub/builder.ex
@ -7,11 +7,115 @@ defmodule Pleroma.Web.ActivityPub.Builder do

  alias Pleroma.Object
  alias Pleroma.User
+  alias Pleroma.Web.ActivityPub.Relay
  alias Pleroma.Web.ActivityPub.Utils
  alias Pleroma.Web.ActivityPub.Visibility

+  require Pleroma.Constants
+
+  @spec emoji_react(User.t(), Object.t(), String.t()) :: {:ok, map(), keyword()}
+  def emoji_react(actor, object, emoji) do
+    with {:ok, data, meta} <- object_action(actor, object) do
+      data =
+        data
+        |> Map.put("content", emoji)
+        |> Map.put("type", "EmojiReact")
+
+      {:ok, data, meta}
+    end
+  end
+
+  @spec undo(User.t(), Activity.t()) :: {:ok, map(), keyword()}
+  def undo(actor, object) do
+    {:ok,
+     %{
+       "id" => Utils.generate_activity_id(),
+       "actor" => actor.ap_id,
+       "type" => "Undo",
+       "object" => object.data["id"],
+       "to" => object.data["to"] || [],
+       "cc" => object.data["cc"] || []
+     }, []}
+  end
+
+  @spec delete(User.t(), String.t()) :: {:ok, map(), keyword()}
+  def delete(actor, object_id) do
+    object = Object.normalize(object_id, false)
+
+    user = !object && User.get_cached_by_ap_id(object_id)
+
+    to =
+      case {object, user} do
+        {%Object{}, _} ->
+          # We are deleting an object, address everyone who was originally mentioned
+          (object.data["to"] || []) ++ (object.data["cc"] || [])
+
+        {_, %User{follower_address: follower_address}} ->
+          # We are deleting a user, address the followers of that user
+          [follower_address]
+      end
+
+    {:ok,
+     %{
+       "id" => Utils.generate_activity_id(),
+       "actor" => actor.ap_id,
+       "object" => object_id,
+       "to" => to,
+       "type" => "Delete"
+     }, []}
+  end
+
+  @spec tombstone(String.t(), String.t()) :: {:ok, map(), keyword()}
+  def tombstone(actor, id) do
+    {:ok,
+     %{
+       "id" => id,
+       "actor" => actor,
+       "type" => "Tombstone"
+     }, []}
+  end
+
  @spec like(User.t(), Object.t()) :: {:ok, map(), keyword()}
  def like(actor, object) do
+    with {:ok, data, meta} <- object_action(actor, object) do
+      data =
+        data
+        |> Map.put("type", "Like")
+
+      {:ok, data, meta}
+    end
+  end
+
+  @spec announce(User.t(), Object.t(), keyword()) :: {:ok, map(), keyword()}
+  def announce(actor, object, options \\ []) do
+    public? = Keyword.get(options, :public, false)
+
+    to =
+      cond do
+        actor.ap_id == Relay.relay_ap_id() ->
+          [actor.follower_address]
+
+        public? ->
+          [actor.follower_address, object.data["actor"], Pleroma.Constants.as_public()]
+
+        true ->
+          [actor.follower_address, object.data["actor"]]
+      end
+
+    {:ok,
+     %{
+       "id" => Utils.generate_activity_id(),
+       "actor" => actor.ap_id,
+       "object" => object.data["id"],
+       "to" => to,
+       "context" => object.data["context"],
+       "type" => "Announce",
+       "published" => Utils.make_date()
+     }, []}
+  end
+
+  @spec object_action(User.t(), Object.t()) :: {:ok, map(), keyword()}
+  defp object_action(actor, object) do
    object_actor = User.get_cached_by_ap_id(object.data["actor"])

    # Address the actor of the object, and our actor's follower collection if the post is public.
@ -33,7 +137,6 @@ def like(actor, object) do
     %{
       "id" => Utils.generate_activity_id(),
       "actor" => actor.ap_id,
-       "type" => "Like",
       "object" => object.data["id"],
       "to" => to,
       "cc" => cc,
--- a/lib/pleroma/web/activity_pub/mrf/steal_emoji_policy.ex
+++ b/lib/pleroma/web/activity_pub/mrf/steal_emoji_policy.ex
@ -0,0 +1,97 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ActivityPub.MRF.StealEmojiPolicy do
+  require Logger
+
+  alias Pleroma.Config
+
+  @moduledoc "Detect new emojis by their shortcode and steals them"
+  @behaviour Pleroma.Web.ActivityPub.MRF
+
+  defp remote_host?(host), do: host != Config.get([Pleroma.Web.Endpoint, :url, :host])
+
+  defp accept_host?(host), do: host in Config.get([:mrf_steal_emoji, :hosts], [])
+
+  defp steal_emoji({shortcode, url}) do
+    url = Pleroma.Web.MediaProxy.url(url)
+    {:ok, response} = Pleroma.HTTP.get(url)
+    size_limit = Config.get([:mrf_steal_emoji, :size_limit], 50_000)
+
+    if byte_size(response.body) <= size_limit do
+      emoji_dir_path =
+        Config.get(
+          [:mrf_steal_emoji, :path],
+          Path.join(Config.get([:instance, :static_dir]), "emoji/stolen")
+        )
+
+      extension =
+        url
+        |> URI.parse()
+        |> Map.get(:path)
+        |> Path.basename()
+        |> Path.extname()
+
+      file_path = Path.join([emoji_dir_path, shortcode <> (extension || ".png")])
+
+      try do
+        :ok = File.write(file_path, response.body)
+
+        shortcode
+      rescue
+        e ->
+          Logger.warn("MRF.StealEmojiPolicy: Failed to write to #{file_path}: #{inspect(e)}")
+          nil
+      end
+    else
+      Logger.debug(
+        "MRF.StealEmojiPolicy: :#{shortcode}: at #{url} (#{byte_size(response.body)} B) over size limit (#{
+          size_limit
+        } B)"
+      )
+
+      nil
+    end
+  rescue
+    e ->
+      Logger.warn("MRF.StealEmojiPolicy: Failed to fetch #{url}: #{inspect(e)}")
+      nil
+  end
+
+  @impl true
+  def filter(%{"object" => %{"emoji" => foreign_emojis, "actor" => actor}} = message) do
+    host = URI.parse(actor).host
+
+    if remote_host?(host) and accept_host?(host) do
+      installed_emoji = Pleroma.Emoji.get_all() |> Enum.map(fn {k, _} -> k end)
+
+      new_emojis =
+        foreign_emojis
+        |> Enum.filter(fn {shortcode, _url} -> shortcode not in installed_emoji end)
+        |> Enum.filter(fn {shortcode, _url} ->
+          reject_emoji? =
+            Config.get([:mrf_steal_emoji, :rejected_shortcodes], [])
+            |> Enum.find(false, fn regex -> String.match?(shortcode, regex) end)
+
+          !reject_emoji?
+        end)
+        |> Enum.map(&steal_emoji(&1))
+        |> Enum.filter(& &1)
+
+      if !Enum.empty?(new_emojis) do
+        Logger.info("Stole new emojis: #{inspect(new_emojis)}")
+        Pleroma.Emoji.reload()
+      end
+    end
+
+    {:ok, message}
+  end
+
+  def filter(message), do: {:ok, message}
+
+  @impl true
+  def describe do
+    {:ok, %{}}
+  end
+end
--- a/lib/pleroma/web/activity_pub/object_validator.ex
+++ b/lib/pleroma/web/activity_pub/object_validator.ex
@ -11,11 +11,36 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidator do

  alias Pleroma.Object
  alias Pleroma.User
+  alias Pleroma.Web.ActivityPub.ObjectValidators.AnnounceValidator
+  alias Pleroma.Web.ActivityPub.ObjectValidators.DeleteValidator
+  alias Pleroma.Web.ActivityPub.ObjectValidators.EmojiReactValidator
  alias Pleroma.Web.ActivityPub.ObjectValidators.LikeValidator
+  alias Pleroma.Web.ActivityPub.ObjectValidators.Types
+  alias Pleroma.Web.ActivityPub.ObjectValidators.UndoValidator

  @spec validate(map(), keyword()) :: {:ok, map(), keyword()} | {:error, any()}
  def validate(object, meta)

+  def validate(%{"type" => "Undo"} = object, meta) do
+    with {:ok, object} <-
+           object
+           |> UndoValidator.cast_and_validate()
+           |> Ecto.Changeset.apply_action(:insert) do
+      object = stringify_keys(object)
+      {:ok, object, meta}
+    end
+  end
+
+  def validate(%{"type" => "Delete"} = object, meta) do
+    with cng <- DeleteValidator.cast_and_validate(object),
+         do_not_federate <- DeleteValidator.do_not_federate?(cng),
+         {:ok, object} <- Ecto.Changeset.apply_action(cng, :insert) do
+      object = stringify_keys(object)
+      meta = Keyword.put(meta, :do_not_federate, do_not_federate)
+      {:ok, object, meta}
+    end
+  end
+
  def validate(%{"type" => "Like"} = object, meta) do
    with {:ok, object} <-
           object |> LikeValidator.cast_and_validate() |> Ecto.Changeset.apply_action(:insert) do
@ -24,14 +49,46 @@ def validate(%{"type" => "Like"} = object, meta) do
    end
  end

+  def validate(%{"type" => "EmojiReact"} = object, meta) do
+    with {:ok, object} <-
+           object
+           |> EmojiReactValidator.cast_and_validate()
+           |> Ecto.Changeset.apply_action(:insert) do
+      object = stringify_keys(object |> Map.from_struct())
+      {:ok, object, meta}
+    end
+  end
+
+  def validate(%{"type" => "Announce"} = object, meta) do
+    with {:ok, object} <-
+           object
+           |> AnnounceValidator.cast_and_validate()
+           |> Ecto.Changeset.apply_action(:insert) do
+      object = stringify_keys(object |> Map.from_struct())
+      {:ok, object, meta}
+    end
+  end
+
+  def stringify_keys(%{__struct__: _} = object) do
+    object
+    |> Map.from_struct()
+    |> stringify_keys
+  end
+
  def stringify_keys(object) do
    object
    |> Map.new(fn {key, val} -> {to_string(key), val} end)
  end

+  def fetch_actor(object) do
+    with {:ok, actor} <- Types.ObjectID.cast(object["actor"]) do
+      User.get_or_fetch_by_ap_id(actor)
+    end
+  end
+
  def fetch_actor_and_object(object) do
-    User.get_or_fetch_by_ap_id(object["actor"])
-    Object.normalize(object["object"])
+    fetch_actor(object)
+    Object.normalize(object["object"], true)
    :ok
  end
 end
--- a/lib/pleroma/web/activity_pub/object_validators/announce_validator.ex
+++ b/lib/pleroma/web/activity_pub/object_validators/announce_validator.ex
@ -0,0 +1,101 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ActivityPub.ObjectValidators.AnnounceValidator do
+  use Ecto.Schema
+
+  alias Pleroma.Object
+  alias Pleroma.User
+  alias Pleroma.Web.ActivityPub.ObjectValidators.Types
+  alias Pleroma.Web.ActivityPub.Utils
+  alias Pleroma.Web.ActivityPub.Visibility
+
+  import Ecto.Changeset
+  import Pleroma.Web.ActivityPub.ObjectValidators.CommonValidations
+
+  require Pleroma.Constants
+
+  @primary_key false
+
+  embedded_schema do
+    field(:id, Types.ObjectID, primary_key: true)
+    field(:type, :string)
+    field(:object, Types.ObjectID)
+    field(:actor, Types.ObjectID)
+    field(:context, :string, autogenerate: {Utils, :generate_context_id, []})
+    field(:to, Types.Recipients, default: [])
+    field(:cc, Types.Recipients, default: [])
+    field(:published, Types.DateTime)
+  end
+
+  def cast_and_validate(data) do
+    data
+    |> cast_data()
+    |> validate_data()
+  end
+
+  def cast_data(data) do
+    %__MODULE__{}
+    |> changeset(data)
+  end
+
+  def changeset(struct, data) do
+    struct
+    |> cast(data, __schema__(:fields))
+    |> fix_after_cast()
+  end
+
+  def fix_after_cast(cng) do
+    cng
+  end
+
+  def validate_data(data_cng) do
+    data_cng
+    |> validate_inclusion(:type, ["Announce"])
+    |> validate_required([:id, :type, :object, :actor, :to, :cc])
+    |> validate_actor_presence()
+    |> validate_object_presence()
+    |> validate_existing_announce()
+    |> validate_announcable()
+  end
+
+  def validate_announcable(cng) do
+    with actor when is_binary(actor) <- get_field(cng, :actor),
+         object when is_binary(object) <- get_field(cng, :object),
+         %User{} = actor <- User.get_cached_by_ap_id(actor),
+         %Object{} = object <- Object.get_cached_by_ap_id(object),
+         false <- Visibility.is_public?(object) do
+      same_actor = object.data["actor"] == actor.ap_id
+      is_public = Pleroma.Constants.as_public() in (get_field(cng, :to) ++ get_field(cng, :cc))
+
+      cond do
+        same_actor && is_public ->
+          cng
+          |> add_error(:actor, "can not announce this object publicly")
+
+        !same_actor ->
+          cng
+          |> add_error(:actor, "can not announce this object")
+
+        true ->
+          cng
+      end
+    else
+      _ -> cng
+    end
+  end
+
+  def validate_existing_announce(cng) do
+    actor = get_field(cng, :actor)
+    object = get_field(cng, :object)
+
+    if actor && object && Utils.get_existing_announce(actor, %{data: %{"id" => object}}) do
+      cng
+      |> add_error(:actor, "already announced this object")
+      |> add_error(:object, "already announced by this actor")
+    else
+      cng
+    end
+  end
+end
--- a/lib/pleroma/web/activity_pub/object_validators/common_validations.ex
+++ b/lib/pleroma/web/activity_pub/object_validators/common_validations.ex
@ -5,10 +5,33 @@
 defmodule Pleroma.Web.ActivityPub.ObjectValidators.CommonValidations do
  import Ecto.Changeset

+  alias Pleroma.Activity
  alias Pleroma.Object
  alias Pleroma.User

-  def validate_actor_presence(cng, field_name \\ :actor) do
+  def validate_recipients_presence(cng, fields \\ [:to, :cc]) do
+    non_empty =
+      fields
+      |> Enum.map(fn field -> get_field(cng, field) end)
+      |> Enum.any?(fn
+        [] -> false
+        _ -> true
+      end)
+
+    if non_empty do
+      cng
+    else
+      fields
+      |> Enum.reduce(cng, fn field, cng ->
+        cng
+        |> add_error(field, "no recipients in any field")
+      end)
+    end
+  end
+
+  def validate_actor_presence(cng, options \\ []) do
+    field_name = Keyword.get(options, :field_name, :actor)
+
    cng
    |> validate_change(field_name, fn field_name, actor ->
      if User.get_cached_by_ap_id(actor) do
@ -19,14 +42,39 @@ def validate_actor_presence(cng, field_name \\ :actor) do
    end)
  end

-  def validate_object_presence(cng, field_name \\ :object) do
+  def validate_object_presence(cng, options \\ []) do
+    field_name = Keyword.get(options, :field_name, :object)
+    allowed_types = Keyword.get(options, :allowed_types, false)
+
    cng
-    |> validate_change(field_name, fn field_name, object ->
-      if Object.get_cached_by_ap_id(object) do
-        []
-      else
-        [{field_name, "can't find object"}]
+    |> validate_change(field_name, fn field_name, object_id ->
+      object = Object.get_cached_by_ap_id(object_id) || Activity.get_by_ap_id(object_id)
+
+      cond do
+        !object ->
+          [{field_name, "can't find object"}]
+
+        object && allowed_types && object.data["type"] not in allowed_types ->
+          [{field_name, "object not in allowed types"}]
+
+        true ->
+          []
      end
    end)
  end
+
+  def validate_object_or_user_presence(cng, options \\ []) do
+    field_name = Keyword.get(options, :field_name, :object)
+    options = Keyword.put(options, :field_name, field_name)
+
+    actor_cng =
+      cng
+      |> validate_actor_presence(options)
+
+    object_cng =
+      cng
+      |> validate_object_presence(options)
+
+    if actor_cng.valid?, do: actor_cng, else: object_cng
+  end
 end
--- a/lib/pleroma/web/activity_pub/object_validators/delete_validator.ex
+++ b/lib/pleroma/web/activity_pub/object_validators/delete_validator.ex
@ -0,0 +1,100 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ActivityPub.ObjectValidators.DeleteValidator do
+  use Ecto.Schema
+
+  alias Pleroma.Activity
+  alias Pleroma.User
+  alias Pleroma.Web.ActivityPub.ObjectValidators.Types
+
+  import Ecto.Changeset
+  import Pleroma.Web.ActivityPub.ObjectValidators.CommonValidations
+
+  @primary_key false
+
+  embedded_schema do
+    field(:id, Types.ObjectID, primary_key: true)
+    field(:type, :string)
+    field(:actor, Types.ObjectID)
+    field(:to, Types.Recipients, default: [])
+    field(:cc, Types.Recipients, default: [])
+    field(:deleted_activity_id, Types.ObjectID)
+    field(:object, Types.ObjectID)
+  end
+
+  def cast_data(data) do
+    %__MODULE__{}
+    |> cast(data, __schema__(:fields))
+  end
+
+  def add_deleted_activity_id(cng) do
+    object =
+      cng
+      |> get_field(:object)
+
+    with %Activity{id: id} <- Activity.get_create_by_object_ap_id(object) do
+      cng
+      |> put_change(:deleted_activity_id, id)
+    else
+      _ -> cng
+    end
+  end
+
+  @deletable_types ~w{
+    Answer
+    Article
+    Audio
+    Event
+    Note
+    Page
+    Question
+    Video
+    Tombstone
+  }
+  def validate_data(cng) do
+    cng
+    |> validate_required([:id, :type, :actor, :to, :cc, :object])
+    |> validate_inclusion(:type, ["Delete"])
+    |> validate_actor_presence()
+    |> validate_deletion_rights()
+    |> validate_object_or_user_presence(allowed_types: @deletable_types)
+    |> add_deleted_activity_id()
+  end
+
+  def do_not_federate?(cng) do
+    !same_domain?(cng)
+  end
+
+  defp same_domain?(cng) do
+    actor_uri =
+      cng
+      |> get_field(:actor)
+      |> URI.parse()
+
+    object_uri =
+      cng
+      |> get_field(:object)
+      |> URI.parse()
+
+    object_uri.host == actor_uri.host
+  end
+
+  def validate_deletion_rights(cng) do
+    actor = User.get_cached_by_ap_id(get_field(cng, :actor))
+
+    if User.superuser?(actor) || same_domain?(cng) do
+      cng
+    else
+      cng
+      |> add_error(:actor, "is not allowed to delete object")
+    end
+  end
+
+  def cast_and_validate(data) do
+    data
+    |> cast_data
+    |> validate_data
+  end
+end
--- a/lib/pleroma/web/activity_pub/object_validators/emoji_react_validator.ex
+++ b/lib/pleroma/web/activity_pub/object_validators/emoji_react_validator.ex
@ -0,0 +1,81 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ActivityPub.ObjectValidators.EmojiReactValidator do
+  use Ecto.Schema
+
+  alias Pleroma.Object
+  alias Pleroma.Web.ActivityPub.ObjectValidators.Types
+
+  import Ecto.Changeset
+  import Pleroma.Web.ActivityPub.ObjectValidators.CommonValidations
+
+  @primary_key false
+
+  embedded_schema do
+    field(:id, Types.ObjectID, primary_key: true)
+    field(:type, :string)
+    field(:object, Types.ObjectID)
+    field(:actor, Types.ObjectID)
+    field(:context, :string)
+    field(:content, :string)
+    field(:to, {:array, :string}, default: [])
+    field(:cc, {:array, :string}, default: [])
+  end
+
+  def cast_and_validate(data) do
+    data
+    |> cast_data()
+    |> validate_data()
+  end
+
+  def cast_data(data) do
+    %__MODULE__{}
+    |> changeset(data)
+  end
+
+  def changeset(struct, data) do
+    struct
+    |> cast(data, __schema__(:fields))
+    |> fix_after_cast()
+  end
+
+  def fix_after_cast(cng) do
+    cng
+    |> fix_context()
+  end
+
+  def fix_context(cng) do
+    object = get_field(cng, :object)
+
+    with nil <- get_field(cng, :context),
+         %Object{data: %{"context" => context}} <- Object.get_cached_by_ap_id(object) do
+      cng
+      |> put_change(:context, context)
+    else
+      _ ->
+        cng
+    end
+  end
+
+  def validate_emoji(cng) do
+    content = get_field(cng, :content)
+
+    if Pleroma.Emoji.is_unicode_emoji?(content) do
+      cng
+    else
+      cng
+      |> add_error(:content, "must be a single character emoji")
+    end
+  end
+
+  def validate_data(data_cng) do
+    data_cng
+    |> validate_inclusion(:type, ["EmojiReact"])
+    |> validate_required([:id, :type, :object, :actor, :context, :to, :cc, :content])
+    |> validate_actor_presence()
+    |> validate_object_presence()
+    |> validate_emoji()
+  end
+end
--- a/lib/pleroma/web/activity_pub/object_validators/like_validator.ex
+++ b/lib/pleroma/web/activity_pub/object_validators/like_validator.ex
@ -20,8 +20,8 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.LikeValidator do
    field(:object, Types.ObjectID)
    field(:actor, Types.ObjectID)
    field(:context, :string)
-    field(:to, {:array, :string}, default: [])
-    field(:cc, {:array, :string}, default: [])
+    field(:to, Types.Recipients, default: [])
+    field(:cc, Types.Recipients, default: [])
  end

  def cast_and_validate(data) do
--- a/lib/pleroma/web/activity_pub/object_validators/types/recipients.ex
+++ b/lib/pleroma/web/activity_pub/object_validators/types/recipients.ex
@ -0,0 +1,34 @@
+defmodule Pleroma.Web.ActivityPub.ObjectValidators.Types.Recipients do
+  use Ecto.Type
+
+  alias Pleroma.Web.ActivityPub.ObjectValidators.Types.ObjectID
+
+  def type, do: {:array, ObjectID}
+
+  def cast(object) when is_binary(object) do
+    cast([object])
+  end
+
+  def cast(data) when is_list(data) do
+    data
+    |> Enum.reduce({:ok, []}, fn element, acc ->
+      case {acc, ObjectID.cast(element)} do
+        {:error, _} -> :error
+        {_, :error} -> :error
+        {{:ok, list}, {:ok, id}} -> {:ok, [id | list]}
+      end
+    end)
+  end
+
+  def cast(_) do
+    :error
+  end
+
+  def dump(data) do
+    {:ok, data}
+  end
+
+  def load(data) do
+    {:ok, data}
+  end
+end
--- a/lib/pleroma/web/activity_pub/object_validators/undo_validator.ex
+++ b/lib/pleroma/web/activity_pub/object_validators/undo_validator.ex
@ -0,0 +1,62 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ActivityPub.ObjectValidators.UndoValidator do
+  use Ecto.Schema
+
+  alias Pleroma.Activity
+  alias Pleroma.Web.ActivityPub.ObjectValidators.Types
+
+  import Ecto.Changeset
+  import Pleroma.Web.ActivityPub.ObjectValidators.CommonValidations
+
+  @primary_key false
+
+  embedded_schema do
+    field(:id, Types.ObjectID, primary_key: true)
+    field(:type, :string)
+    field(:object, Types.ObjectID)
+    field(:actor, Types.ObjectID)
+    field(:to, {:array, :string}, default: [])
+    field(:cc, {:array, :string}, default: [])
+  end
+
+  def cast_and_validate(data) do
+    data
+    |> cast_data()
+    |> validate_data()
+  end
+
+  def cast_data(data) do
+    %__MODULE__{}
+    |> changeset(data)
+  end
+
+  def changeset(struct, data) do
+    struct
+    |> cast(data, __schema__(:fields))
+  end
+
+  def validate_data(data_cng) do
+    data_cng
+    |> validate_inclusion(:type, ["Undo"])
+    |> validate_required([:id, :type, :object, :actor, :to, :cc])
+    |> validate_actor_presence()
+    |> validate_object_presence()
+    |> validate_undo_rights()
+  end
+
+  def validate_undo_rights(cng) do
+    actor = get_field(cng, :actor)
+    object = get_field(cng, :object)
+
+    with %Activity{data: %{"actor" => object_actor}} <- Activity.get_by_ap_id(object),
+         true <- object_actor != actor do
+      cng
+      |> add_error(:actor, "not the same as object actor")
+    else
+      _ -> cng
+    end
+  end
+end
--- a/lib/pleroma/web/activity_pub/pipeline.ex
+++ b/lib/pleroma/web/activity_pub/pipeline.ex
@ -4,6 +4,7 @@

 defmodule Pleroma.Web.ActivityPub.Pipeline do
  alias Pleroma.Activity
+  alias Pleroma.Config
  alias Pleroma.Object
  alias Pleroma.Repo
  alias Pleroma.Web.ActivityPub.ActivityPub
@ -44,7 +45,9 @@ defp maybe_federate(%Object{}, _), do: {:ok, :not_federated}

  defp maybe_federate(%Activity{} = activity, meta) do
    with {:ok, local} <- Keyword.fetch(meta, :local) do
-      if local do
+      do_not_federate = meta[:do_not_federate] || !Config.get([:instance, :federating])
+
+      if !do_not_federate && local do
        Federator.publish(activity)
        {:ok, :federated}
      else
--- a/lib/pleroma/web/activity_pub/relay.ex
+++ b/lib/pleroma/web/activity_pub/relay.ex
@ -4,9 +4,10 @@

 defmodule Pleroma.Web.ActivityPub.Relay do
  alias Pleroma.Activity
-  alias Pleroma.Object
  alias Pleroma.User
  alias Pleroma.Web.ActivityPub.ActivityPub
+  alias Pleroma.Web.ActivityPub.Visibility
+  alias Pleroma.Web.CommonAPI
  require Logger

  @relay_nickname "relay"
@ -48,11 +49,11 @@ def unfollow(target_instance) do
    end
  end

-  @spec publish(any()) :: {:ok, Activity.t(), Object.t()} | {:error, any()}
+  @spec publish(any()) :: {:ok, Activity.t()} | {:error, any()}
  def publish(%Activity{data: %{"type" => "Create"}} = activity) do
    with %User{} = user <- get_actor(),
-         %Object{} = object <- Object.normalize(activity) do
-      ActivityPub.announce(user, object, nil, true, false)
+         true <- Visibility.is_public?(activity) do
+      CommonAPI.repeat(activity.id, user)
    else
      error -> format_error(error)
    end
--- a/lib/pleroma/web/activity_pub/side_effects.ex
+++ b/lib/pleroma/web/activity_pub/side_effects.ex
@ -5,8 +5,12 @@ defmodule Pleroma.Web.ActivityPub.SideEffects do
  liked object, a `Follow` activity will add the user to the follower
  collection, and so on.
  """
+  alias Pleroma.Activity
  alias Pleroma.Notification
  alias Pleroma.Object
+  alias Pleroma.Repo
+  alias Pleroma.User
+  alias Pleroma.Web.ActivityPub.ActivityPub
  alias Pleroma.Web.ActivityPub.Utils

  def handle(object, meta \\ [])
@ -23,8 +27,125 @@ def handle(%{data: %{"type" => "Like"}} = object, meta) do
    {:ok, object, meta}
  end

+  # Tasks this handles:
+  # - Add announce to object
+  # - Set up notification
+  # - Stream out the announce
+  def handle(%{data: %{"type" => "Announce"}} = object, meta) do
+    announced_object = Object.get_by_ap_id(object.data["object"])
+    user = User.get_cached_by_ap_id(object.data["actor"])
+
+    Utils.add_announce_to_object(object, announced_object)
+
+    if !User.is_internal_user?(user) do
+      Notification.create_notifications(object)
+      ActivityPub.stream_out(object)
+    end
+
+    {:ok, object, meta}
+  end
+
+  def handle(%{data: %{"type" => "Undo", "object" => undone_object}} = object, meta) do
+    with undone_object <- Activity.get_by_ap_id(undone_object),
+         :ok <- handle_undoing(undone_object) do
+      {:ok, object, meta}
+    end
+  end
+
+  # Tasks this handles:
+  # - Add reaction to object
+  # - Set up notification
+  def handle(%{data: %{"type" => "EmojiReact"}} = object, meta) do
+    reacted_object = Object.get_by_ap_id(object.data["object"])
+    Utils.add_emoji_reaction_to_object(object, reacted_object)
+
+    Notification.create_notifications(object)
+
+    {:ok, object, meta}
+  end
+
+  # Tasks this handles:
+  # - Delete and unpins the create activity
+  # - Replace object with Tombstone
+  # - Set up notification
+  # - Reduce the user note count
+  # - Reduce the reply count
+  # - Stream out the activity
+  def handle(%{data: %{"type" => "Delete", "object" => deleted_object}} = object, meta) do
+    deleted_object =
+      Object.normalize(deleted_object, false) || User.get_cached_by_ap_id(deleted_object)
+
+    result =
+      case deleted_object do
+        %Object{} ->
+          with {:ok, deleted_object, activity} <- Object.delete(deleted_object),
+               %User{} = user <- User.get_cached_by_ap_id(deleted_object.data["actor"]) do
+            User.remove_pinnned_activity(user, activity)
+
+            {:ok, user} = ActivityPub.decrease_note_count_if_public(user, deleted_object)
+
+            if in_reply_to = deleted_object.data["inReplyTo"] do
+              Object.decrease_replies_count(in_reply_to)
+            end
+
+            ActivityPub.stream_out(object)
+            ActivityPub.stream_out_participations(deleted_object, user)
+            :ok
+          end
+
+        %User{} ->
+          with {:ok, _} <- User.delete(deleted_object) do
+            :ok
+          end
+      end
+
+    if result == :ok do
+      Notification.create_notifications(object)
+      {:ok, object, meta}
+    else
+      {:error, result}
+    end
+  end
+
  # Nothing to do
  def handle(object, meta) do
    {:ok, object, meta}
  end
+
+  def handle_undoing(%{data: %{"type" => "Like"}} = object) do
+    with %Object{} = liked_object <- Object.get_by_ap_id(object.data["object"]),
+         {:ok, _} <- Utils.remove_like_from_object(object, liked_object),
+         {:ok, _} <- Repo.delete(object) do
+      :ok
+    end
+  end
+
+  def handle_undoing(%{data: %{"type" => "EmojiReact"}} = object) do
+    with %Object{} = reacted_object <- Object.get_by_ap_id(object.data["object"]),
+         {:ok, _} <- Utils.remove_emoji_reaction_from_object(object, reacted_object),
+         {:ok, _} <- Repo.delete(object) do
+      :ok
+    end
+  end
+
+  def handle_undoing(%{data: %{"type" => "Announce"}} = object) do
+    with %Object{} = liked_object <- Object.get_by_ap_id(object.data["object"]),
+         {:ok, _} <- Utils.remove_announce_from_object(object, liked_object),
+         {:ok, _} <- Repo.delete(object) do
+      :ok
+    end
+  end
+
+  def handle_undoing(
+        %{data: %{"type" => "Block", "actor" => blocker, "object" => blocked}} = object
+      ) do
+    with %User{} = blocker <- User.get_cached_by_ap_id(blocker),
+         %User{} = blocked <- User.get_cached_by_ap_id(blocked),
+         {:ok, _} <- User.unblock(blocker, blocked),
+         {:ok, _} <- Repo.delete(object) do
+      :ok
+    end
+  end
+
+  def handle_undoing(object), do: {:error, ["don't know how to handle", object]}
 end
--- a/lib/pleroma/web/activity_pub/transmogrifier.ex
+++ b/lib/pleroma/web/activity_pub/transmogrifier.ex
@ -14,7 +14,9 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
  alias Pleroma.Repo
  alias Pleroma.User
  alias Pleroma.Web.ActivityPub.ActivityPub
+  alias Pleroma.Web.ActivityPub.Builder
  alias Pleroma.Web.ActivityPub.ObjectValidator
+  alias Pleroma.Web.ActivityPub.ObjectValidators.Types
  alias Pleroma.Web.ActivityPub.Pipeline
  alias Pleroma.Web.ActivityPub.Utils
  alias Pleroma.Web.ActivityPub.Visibility
@ -590,6 +592,9 @@ def handle_incoming(
         {:ok, follow_activity} <- Utils.update_follow_state_for_all(follow_activity, "accept"),
         %User{local: true} = follower <- User.get_cached_by_ap_id(follow_activity.data["actor"]),
         {:ok, _relationship} <- FollowingRelationship.update(follower, followed, :follow_accept) do
+      User.update_follower_count(followed)
+      User.update_following_count(follower)
+
      ActivityPub.accept(%{
        to: follow_activity.data["to"],
        type: "Accept",
@ -599,7 +604,8 @@ def handle_incoming(
        activity_id: id
      })
    else
-      _e -> :error
+      _e ->
+        :error
    end
  end

@ -656,7 +662,8 @@ def handle_incoming(
    |> handle_incoming(options)
  end

-  def handle_incoming(%{"type" => "Like"} = data, _options) do
+  def handle_incoming(%{"type" => type} = data, _options)
+      when type in ["Like", "EmojiReact", "Announce"] do
    with :ok <- ObjectValidator.fetch_actor_and_object(data),
         {:ok, activity, _meta} <-
           Pipeline.common_pipeline(data, local: false) do
@ -666,42 +673,6 @@ def handle_incoming(%{"type" => "Like"} = data, _options) do
    end
  end

-  def handle_incoming(
-        %{
-          "type" => "EmojiReact",
-          "object" => object_id,
-          "actor" => _actor,
-          "id" => id,
-          "content" => emoji
-        } = data,
-        _options
-      ) do
-    with actor <- Containment.get_actor(data),
-         {:ok, %User{} = actor} <- User.get_or_fetch_by_ap_id(actor),
-         {:ok, object} <- get_obj_helper(object_id),
-         {:ok, activity, _object} <-
-           ActivityPub.react_with_emoji(actor, object, emoji, activity_id: id, local: false) do
-      {:ok, activity}
-    else
-      _e -> :error
-    end
-  end
-
-  def handle_incoming(
-        %{"type" => "Announce", "object" => object_id, "actor" => _actor, "id" => id} = data,
-        _options
-      ) do
-    with actor <- Containment.get_actor(data),
-         {:ok, %User{} = actor} <- User.get_or_fetch_by_ap_id(actor),
-         {:ok, object} <- get_embedded_obj_helper(object_id, actor),
-         public <- Visibility.is_public?(data),
-         {:ok, activity, _object} <- ActivityPub.announce(actor, object, id, false, public) do
-      {:ok, activity}
-    else
-      _e -> :error
-    end
-  end
-
  def handle_incoming(
        %{"type" => "Update", "object" => %{"type" => object_type} = object, "actor" => actor_id} =
          data,
@ -735,55 +706,25 @@ def handle_incoming(
    end
  end

-  # TODO: We presently assume that any actor on the same origin domain as the object being
-  # deleted has the rights to delete that object.  A better way to validate whether or not
-  # the object should be deleted is to refetch the object URI, which should return either
-  # an error or a tombstone.  This would allow us to verify that a deletion actually took
-  # place.
  def handle_incoming(
-        %{"type" => "Delete", "object" => object_id, "actor" => actor, "id" => id} = data,
+        %{"type" => "Delete"} = data,
        _options
      ) do
-    object_id = Utils.get_ap_id(object_id)
-
-    with actor <- Containment.get_actor(data),
-         {:ok, %User{} = actor} <- User.get_or_fetch_by_ap_id(actor),
-         {:ok, object} <- get_obj_helper(object_id),
-         :ok <- Containment.contain_origin(actor.ap_id, object.data),
-         {:ok, activity} <-
-           ActivityPub.delete(object, local: false, activity_id: id, actor: actor.ap_id) do
+    with {:ok, activity, _} <- Pipeline.common_pipeline(data, local: false) do
      {:ok, activity}
    else
-      nil ->
-        case User.get_cached_by_ap_id(object_id) do
-          %User{ap_id: ^actor} = user ->
-            User.delete(user)
-
-          nil ->
-            :error
+      {:error, {:validate_object, _}} = e ->
+        # Check if we have a create activity for this
+        with {:ok, object_id} <- Types.ObjectID.cast(data["object"]),
+             %Activity{data: %{"actor" => actor}} <-
+               Activity.create_by_object_ap_id(object_id) |> Repo.one(),
+             # We have one, insert a tombstone and retry
+             {:ok, tombstone_data, _} <- Builder.tombstone(actor, object_id),
+             {:ok, _tombstone} <- Object.create(tombstone_data) do
+          handle_incoming(data)
+        else
+          _ -> e
        end
-
-      _e ->
-        :error
-    end
-  end
-
-  def handle_incoming(
-        %{
-          "type" => "Undo",
-          "object" => %{"type" => "Announce", "object" => object_id},
-          "actor" => _actor,
-          "id" => id
-        } = data,
-        _options
-      ) do
-    with actor <- Containment.get_actor(data),
-         {:ok, %User{} = actor} <- User.get_or_fetch_by_ap_id(actor),
-         {:ok, object} <- get_obj_helper(object_id),
-         {:ok, activity, _} <- ActivityPub.unannounce(actor, object, id, false) do
-      {:ok, activity}
-    else
-      _e -> :error
    end
  end

@ -809,75 +750,13 @@ def handle_incoming(
  def handle_incoming(
        %{
          "type" => "Undo",
-          "object" => %{"type" => "EmojiReact", "id" => reaction_activity_id},
-          "actor" => _actor,
-          "id" => id
+          "object" => %{"type" => type}
        } = data,
        _options
-      ) do
-    with actor <- Containment.get_actor(data),
-         {:ok, %User{} = actor} <- User.get_or_fetch_by_ap_id(actor),
-         {:ok, activity, _} <-
-           ActivityPub.unreact_with_emoji(actor, reaction_activity_id,
-             activity_id: id,
-             local: false
-           ) do
+      )
+      when type in ["Like", "EmojiReact", "Announce", "Block"] do
+    with {:ok, activity, _} <- Pipeline.common_pipeline(data, local: false) do
      {:ok, activity}
-    else
-      _e -> :error
-    end
-  end
-
-  def handle_incoming(
-        %{
-          "type" => "Undo",
-          "object" => %{"type" => "Block", "object" => blocked},
-          "actor" => blocker,
-          "id" => id
-        } = _data,
-        _options
-      ) do
-    with %User{local: true} = blocked <- User.get_cached_by_ap_id(blocked),
-         {:ok, %User{} = blocker} <- User.get_or_fetch_by_ap_id(blocker),
-         {:ok, activity} <- ActivityPub.unblock(blocker, blocked, id, false) do
-      User.unblock(blocker, blocked)
-      {:ok, activity}
-    else
-      _e -> :error
-    end
-  end
-
-  def handle_incoming(
-        %{"type" => "Block", "object" => blocked, "actor" => blocker, "id" => id} = _data,
-        _options
-      ) do
-    with %User{local: true} = blocked = User.get_cached_by_ap_id(blocked),
-         {:ok, %User{} = blocker} = User.get_or_fetch_by_ap_id(blocker),
-         {:ok, activity} <- ActivityPub.block(blocker, blocked, id, false) do
-      User.unfollow(blocker, blocked)
-      User.block(blocker, blocked)
-      {:ok, activity}
-    else
-      _e -> :error
-    end
-  end
-
-  def handle_incoming(
-        %{
-          "type" => "Undo",
-          "object" => %{"type" => "Like", "object" => object_id},
-          "actor" => _actor,
-          "id" => id
-        } = data,
-        _options
-      ) do
-    with actor <- Containment.get_actor(data),
-         {:ok, %User{} = actor} <- User.get_or_fetch_by_ap_id(actor),
-         {:ok, object} <- get_obj_helper(object_id),
-         {:ok, activity, _, _} <- ActivityPub.unlike(actor, object, id, false) do
-      {:ok, activity}
-    else
-      _e -> :error
    end
  end

@ -899,6 +778,21 @@ def handle_incoming(
    end
  end

+  def handle_incoming(
+        %{"type" => "Block", "object" => blocked, "actor" => blocker, "id" => id} = _data,
+        _options
+      ) do
+    with %User{local: true} = blocked = User.get_cached_by_ap_id(blocked),
+         {:ok, %User{} = blocker} = User.get_or_fetch_by_ap_id(blocker),
+         {:ok, activity} <- ActivityPub.block(blocker, blocked, id, false) do
+      User.unfollow(blocker, blocked)
+      User.block(blocker, blocked)
+      {:ok, activity}
+    else
+      _e -> :error
+    end
+  end
+
  def handle_incoming(
        %{
          "type" => "Move",
@ -1151,10 +1045,14 @@ def add_hashtags(object) do
    Map.put(object, "tag", tags)
  end

+  # TODO These should be added on our side on insertion, it doesn't make much
+  # sense to regenerate these all the time
  def add_mention_tags(object) do
-    {enabled_receivers, disabled_receivers} = Utils.get_notified_from_object(object)
-    potential_receivers = enabled_receivers ++ disabled_receivers
-    mentions = Enum.map(potential_receivers, &build_mention_tag/1)
+    to = object["to"] || []
+    cc = object["cc"] || []
+    mentioned = User.get_users_from_set(to ++ cc, local_only: false)
+
+    mentions = Enum.map(mentioned, &build_mention_tag/1)

    tags = object["tag"] || []
    Map.put(object, "tag", tags ++ mentions)
@ -1195,6 +1093,10 @@ def set_conversation(object) do
    Map.put(object, "conversation", object["context"])
  end

+  def set_sensitive(%{"sensitive" => true} = object) do
+    object
+  end
+
  def set_sensitive(object) do
    tags = object["tag"] || []
    Map.put(object, "sensitive", "nsfw" in tags)
--- a/lib/pleroma/web/activity_pub/utils.ex
+++ b/lib/pleroma/web/activity_pub/utils.ex
@ -6,6 +6,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
  alias Ecto.Changeset
  alias Ecto.UUID
  alias Pleroma.Activity
+  alias Pleroma.Config
  alias Pleroma.Notification
  alias Pleroma.Object
  alias Pleroma.Repo
@ -169,8 +170,11 @@ def create_context(context) do
  Enqueues an activity for federation if it's local
  """
  @spec maybe_federate(any()) :: :ok
-  def maybe_federate(%Activity{local: true} = activity) do
-    if Pleroma.Config.get!([:instance, :federating]) do
+  def maybe_federate(%Activity{local: true, data: %{"type" => type}} = activity) do
+    outgoing_blocks = Config.get([:activitypub, :outgoing_blocks])
+
+    with true <- Config.get!([:instance, :federating]),
+         true <- type != "Block" || outgoing_blocks do
      Pleroma.Web.Federator.publish(activity)
    end

@ -512,7 +516,7 @@ def get_latest_reaction(internal_activity_id, %{ap_id: ap_id}, emoji) do
  #### Announce-related helpers

  @doc """
-  Retruns an existing announce activity if the notice has already been announced
+  Returns an existing announce activity if the notice has already been announced
  """
  @spec get_existing_announce(String.t(), map()) :: Activity.t() | nil
  def get_existing_announce(actor, %{data: %{"id" => ap_id}}) do
@ -562,45 +566,6 @@ def make_announce_data(
    |> maybe_put("id", activity_id)
  end

-  @doc """
-  Make unannounce activity data for the given actor and object
-  """
-  def make_unannounce_data(
-        %User{ap_id: ap_id} = user,
-        %Activity{data: %{"context" => context, "object" => object}} = activity,
-        activity_id
-      ) do
-    object = Object.normalize(object)
-
-    %{
-      "type" => "Undo",
-      "actor" => ap_id,
-      "object" => activity.data,
-      "to" => [user.follower_address, object.data["actor"]],
-      "cc" => [Pleroma.Constants.as_public()],
-      "context" => context
-    }
-    |> maybe_put("id", activity_id)
-  end
-
-  def make_unlike_data(
-        %User{ap_id: ap_id} = user,
-        %Activity{data: %{"context" => context, "object" => object}} = activity,
-        activity_id
-      ) do
-    object = Object.normalize(object)
-
-    %{
-      "type" => "Undo",
-      "actor" => ap_id,
-      "object" => activity.data,
-      "to" => [user.follower_address, object.data["actor"]],
-      "cc" => [Pleroma.Constants.as_public()],
-      "context" => context
-    }
-    |> maybe_put("id", activity_id)
-  end
-
  def make_undo_data(
        %User{ap_id: actor, follower_address: follower_address},
        %Activity{
@ -688,16 +653,6 @@ def make_block_data(blocker, blocked, activity_id) do
    |> maybe_put("id", activity_id)
  end

-  def make_unblock_data(blocker, blocked, block_activity, activity_id) do
-    %{
-      "type" => "Undo",
-      "actor" => blocker.ap_id,
-      "to" => [blocked.ap_id],
-      "object" => block_activity.data
-    }
-    |> maybe_put("id", activity_id)
-  end
-
  #### Create-related helpers

  def make_create_data(params, additional) do
--- a/lib/pleroma/web/admin_api/controllers/admin_api_controller.ex
+++ b/lib/pleroma/web/admin_api/controllers/admin_api_controller.ex
@ -10,6 +10,7 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
  alias Pleroma.Activity
  alias Pleroma.Config
  alias Pleroma.ConfigDB
+  alias Pleroma.MFA
  alias Pleroma.ModerationLog
  alias Pleroma.Plugs.OAuthScopesPlug
  alias Pleroma.ReportNote
@ -17,8 +18,11 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
  alias Pleroma.User
  alias Pleroma.UserInviteToken
  alias Pleroma.Web.ActivityPub.ActivityPub
+  alias Pleroma.Web.ActivityPub.Builder
+  alias Pleroma.Web.ActivityPub.Pipeline
  alias Pleroma.Web.ActivityPub.Relay
  alias Pleroma.Web.ActivityPub.Utils
+  alias Pleroma.Web.AdminAPI
  alias Pleroma.Web.AdminAPI.AccountView
  alias Pleroma.Web.AdminAPI.ConfigView
  alias Pleroma.Web.AdminAPI.ModerationLogView
@ -27,14 +31,14 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
  alias Pleroma.Web.AdminAPI.Search
  alias Pleroma.Web.CommonAPI
  alias Pleroma.Web.Endpoint
+  alias Pleroma.Web.MastodonAPI
  alias Pleroma.Web.MastodonAPI.AppView
-  alias Pleroma.Web.MastodonAPI.StatusView
  alias Pleroma.Web.OAuth.App
  alias Pleroma.Web.Router

  require Logger

-  @descriptions_json Pleroma.Docs.JSON.compile()
+  @descriptions Pleroma.Docs.JSON.compile()
  @users_page_size 50

  plug(
@ -59,6 +63,7 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
           :right_add,
           :right_add_multiple,
           :right_delete,
+           :disable_mfa,
           :right_delete_multiple,
           :update_user_credentials
         ]
@ -93,13 +98,7 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
  plug(
    OAuthScopesPlug,
    %{scopes: ["read:statuses"], admin: true}
-    when action in [:list_statuses, :list_user_statuses, :list_instance_statuses]
-  )
-
-  plug(
-    OAuthScopesPlug,
-    %{scopes: ["write:statuses"], admin: true}
-    when action in [:status_update, :status_delete]
+    when action in [:list_user_statuses, :list_instance_statuses]
  )

  plug(
@ -131,25 +130,22 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
         ]
  )

-  action_fallback(:errors)
+  action_fallback(AdminAPI.FallbackController)

-  def user_delete(%{assigns: %{user: admin}} = conn, %{"nickname" => nickname}) do
-    user = User.get_cached_by_nickname(nickname)
-    User.delete(user)
-
-    ModerationLog.insert_log(%{
-      actor: admin,
-      subject: [user],
-      action: "delete"
-    })
-
-    conn
-    |> json(nickname)
+  def user_delete(conn, %{"nickname" => nickname}) do
+    user_delete(conn, %{"nicknames" => [nickname]})
  end

  def user_delete(%{assigns: %{user: admin}} = conn, %{"nicknames" => nicknames}) do
-    users = nicknames |> Enum.map(&User.get_cached_by_nickname/1)
-    User.delete(users)
+    users =
+      nicknames
+      |> Enum.map(&User.get_cached_by_nickname/1)
+
+    users
+    |> Enum.each(fn user ->
+      {:ok, delete_data, _} = Builder.delete(admin, user.ap_id)
+      Pipeline.common_pipeline(delete_data, local: true)
+    end)

    ModerationLog.insert_log(%{
      actor: admin,
@ -279,8 +275,8 @@ def list_instance_statuses(conn, %{"instance" => instance} = params) do
      })

    conn
-    |> put_view(Pleroma.Web.AdminAPI.StatusView)
-    |> render("index.json", %{activities: activities, as: :activity, skip_relationships: false})
+    |> put_view(AdminAPI.StatusView)
+    |> render("index.json", %{activities: activities, as: :activity})
  end

  def list_user_statuses(conn, %{"nickname" => nickname} = params) do
@ -298,8 +294,8 @@ def list_user_statuses(conn, %{"nickname" => nickname} = params) do
        })

      conn
-      |> put_view(StatusView)
-      |> render("index.json", %{activities: activities, as: :activity, skip_relationships: false})
+      |> put_view(MastodonAPI.StatusView)
+      |> render("index.json", %{activities: activities, as: :activity})
    else
      _ -> {:error, :not_found}
    end
@ -392,29 +388,12 @@ def list_users(conn, params) do
      email: params["email"]
    }

-    with {:ok, users, count} <- Search.user(Map.merge(search_params, filters)),
-         {:ok, users, count} <- filter_service_users(users, count),
-         do:
-           conn
-           |> json(
-             AccountView.render("index.json",
-               users: users,
-               count: count,
-               page_size: page_size
-             )
-           )
-  end
-
-  defp filter_service_users(users, count) do
-    filtered_users = Enum.reject(users, &service_user?/1)
-    count = if Enum.any?(users, &service_user?/1), do: length(filtered_users), else: count
-
-    {:ok, filtered_users, count}
-  end
-
-  defp service_user?(user) do
-    String.match?(user.ap_id, ~r/.*\/relay$/) or
-      String.match?(user.ap_id, ~r/.*\/internal\/fetch$/)
+    with {:ok, users, count} <- Search.user(Map.merge(search_params, filters)) do
+      json(
+        conn,
+        AccountView.render("index.json", users: users, count: count, page_size: page_size)
+      )
+    end
  end

  @filters ~w(local external active deactivated is_admin is_moderator)
@ -612,16 +591,10 @@ def email_invite(%{assigns: %{user: user}} = conn, %{"email" => email} = params)
      json_response(conn, :no_content, "")
    else
      {:registrations_open, _} ->
-        errors(
-          conn,
-          {:error, "To send invites you need to set the `registrations_open` option to false."}
-        )
+        {:error, "To send invites you need to set the `registrations_open` option to false."}

      {:invites_enabled, _} ->
-        errors(
-          conn,
-          {:error, "To send invites you need to set the `invites_enabled` option to true."}
-        )
+        {:error, "To send invites you need to set the `invites_enabled` option to true."}
    end
  end

@ -692,6 +665,18 @@ def force_password_reset(%{assigns: %{user: admin}} = conn, %{"nicknames" => nic
    json_response(conn, :no_content, "")
  end

+  @doc "Disable mfa for user's account."
+  def disable_mfa(conn, %{"nickname" => nickname}) do
+    case User.get_by_nickname(nickname) do
+      %User{} = user ->
+        MFA.disable(user)
+        json(conn, nickname)
+
+      _ ->
+        {:error, :not_found}
+    end
+  end
+
  @doc "Show a given user's credentials"
  def show_user_credentials(%{assigns: %{user: admin}} = conn, %{"nickname" => nickname}) do
    with %User{} = user <- User.get_cached_by_nickname_or_id(nickname) do
@ -708,7 +693,7 @@ def update_user_credentials(
        %{assigns: %{user: admin}} = conn,
        %{"nickname" => nickname} = params
      ) do
-    with {_, user} <- {:user, User.get_cached_by_nickname(nickname)},
+    with {_, %User{} = user} <- {:user, User.get_cached_by_nickname(nickname)},
         {:ok, _user} <-
           User.update_as_admin(user, params) do
      ModerationLog.insert_log(%{
@ -730,11 +715,12 @@ def update_user_credentials(
      json(conn, %{status: "success"})
    else
      {:error, changeset} ->
-        {_, {error, _}} = Enum.at(changeset.errors, 0)
-        json(conn, %{error: "New password #{error}."})
+        errors = Map.new(changeset.errors, fn {key, {error, _}} -> {key, error} end)
+
+        json(conn, %{errors: errors})

      _ ->
-        json(conn, %{error: "Unable to change password."})
+        json(conn, %{error: "Unable to update user."})
    end
  end

@ -817,56 +803,6 @@ def report_notes_delete(%{assigns: %{user: user}} = conn, %{
    end
  end

-  def list_statuses(%{assigns: %{user: _admin}} = conn, params) do
-    godmode = params["godmode"] == "true" || params["godmode"] == true
-    local_only = params["local_only"] == "true" || params["local_only"] == true
-    with_reblogs = params["with_reblogs"] == "true" || params["with_reblogs"] == true
-    {page, page_size} = page_params(params)
-
-    activities =
-      ActivityPub.fetch_statuses(nil, %{
-        "godmode" => godmode,
-        "local_only" => local_only,
-        "limit" => page_size,
-        "offset" => (page - 1) * page_size,
-        "exclude_reblogs" => !with_reblogs && "true"
-      })
-
-    conn
-    |> put_view(Pleroma.Web.AdminAPI.StatusView)
-    |> render("index.json", %{activities: activities, as: :activity, skip_relationships: false})
-  end
-
-  def status_update(%{assigns: %{user: admin}} = conn, %{"id" => id} = params) do
-    with {:ok, activity} <- CommonAPI.update_activity_scope(id, params) do
-      {:ok, sensitive} = Ecto.Type.cast(:boolean, params["sensitive"])
-
-      ModerationLog.insert_log(%{
-        action: "status_update",
-        actor: admin,
-        subject: activity,
-        sensitive: sensitive,
-        visibility: params["visibility"]
-      })
-
-      conn
-      |> put_view(StatusView)
-      |> render("show.json", %{activity: activity})
-    end
-  end
-
-  def status_delete(%{assigns: %{user: user}} = conn, %{"id" => id}) do
-    with {:ok, %Activity{}} <- CommonAPI.delete(id, user) do
-      ModerationLog.insert_log(%{
-        action: "status_delete",
-        actor: user,
-        subject_id: id
-      })
-
-      json(conn, %{})
-    end
-  end
-
  def list_log(conn, params) do
    {page, page_size} = page_params(params)

@ -886,13 +822,13 @@ def list_log(conn, params) do
  end

  def config_descriptions(conn, _params) do
-    conn
-    |> Plug.Conn.put_resp_content_type("application/json")
-    |> Plug.Conn.send_resp(200, @descriptions_json)
+    descriptions = Enum.filter(@descriptions, &whitelisted_config?/1)
+
+    json(conn, descriptions)
  end

  def config_show(conn, %{"only_db" => true}) do
-    with :ok <- configurable_from_database(conn) do
+    with :ok <- configurable_from_database() do
      configs = Pleroma.Repo.all(ConfigDB)

      conn
@ -902,7 +838,7 @@ def config_show(conn, %{"only_db" => true}) do
  end

  def config_show(conn, _params) do
-    with :ok <- configurable_from_database(conn) do
+    with :ok <- configurable_from_database() do
      configs = ConfigDB.get_all_as_keyword()

      merged =
@ -941,9 +877,11 @@ def config_show(conn, _params) do
  end

  def config_update(conn, %{"configs" => configs}) do
-    with :ok <- configurable_from_database(conn) do
+    with :ok <- configurable_from_database() do
      {_errors, results} =
-        Enum.map(configs, fn
+        configs
+        |> Enum.filter(&whitelisted_config?/1)
+        |> Enum.map(fn
          %{"group" => group, "key" => key, "delete" => true} = params ->
            ConfigDB.delete(%{group: group, key: key, subkeys: params["subkeys"]})

@ -983,7 +921,7 @@ def config_update(conn, %{"configs" => configs}) do
  end

  def restart(conn, _params) do
-    with :ok <- configurable_from_database(conn) do
+    with :ok <- configurable_from_database() do
      Restarter.Pleroma.restart(Config.get(:env), 50)

      json(conn, %{})
@ -994,17 +932,36 @@ def need_reboot(conn, _params) do
    json(conn, %{need_reboot: Restarter.Pleroma.need_reboot?()})
  end

-  defp configurable_from_database(conn) do
+  defp configurable_from_database do
    if Config.get(:configurable_from_database) do
      :ok
    else
-      errors(
-        conn,
-        {:error, "To use this endpoint you need to enable configuration from database."}
-      )
+      {:error, "To use this endpoint you need to enable configuration from database."}
    end
  end

+  defp whitelisted_config?(group, key) do
+    if whitelisted_configs = Config.get(:database_config_whitelist) do
+      Enum.any?(whitelisted_configs, fn
+        {whitelisted_group} ->
+          group == inspect(whitelisted_group)
+
+        {whitelisted_group, whitelisted_key} ->
+          group == inspect(whitelisted_group) && key == inspect(whitelisted_key)
+      end)
+    else
+      true
+    end
+  end
+
+  defp whitelisted_config?(%{"group" => group, "key" => key}) do
+    whitelisted_config?(group, key)
+  end
+
+  defp whitelisted_config?(%{:group => group} = config) do
+    whitelisted_config?(group, config[:key])
+  end
+
  def reload_emoji(conn, _params) do
    Pleroma.Emoji.reload()

@ -1123,30 +1080,6 @@ def stats(conn, _) do
    |> json(%{"status_visibility" => count})
  end

-  defp errors(conn, {:error, :not_found}) do
-    conn
-    |> put_status(:not_found)
-    |> json(dgettext("errors", "Not found"))
-  end
-
-  defp errors(conn, {:error, reason}) do
-    conn
-    |> put_status(:bad_request)
-    |> json(reason)
-  end
-
-  defp errors(conn, {:param_cast, _}) do
-    conn
-    |> put_status(:bad_request)
-    |> json(dgettext("errors", "Invalid parameters"))
-  end
-
-  defp errors(conn, _) do
-    conn
-    |> put_status(:internal_server_error)
-    |> json(dgettext("errors", "Something went wrong"))
-  end
-
  defp page_params(params) do
    {get_page(params["page"]), get_page_size(params["page_size"])}
  end
--- a/lib/pleroma/web/admin_api/controllers/fallback_controller.ex
+++ b/lib/pleroma/web/admin_api/controllers/fallback_controller.ex
@ -0,0 +1,31 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.AdminAPI.FallbackController do
+  use Pleroma.Web, :controller
+
+  def call(conn, {:error, :not_found}) do
+    conn
+    |> put_status(:not_found)
+    |> json(%{error: dgettext("errors", "Not found")})
+  end
+
+  def call(conn, {:error, reason}) do
+    conn
+    |> put_status(:bad_request)
+    |> json(%{error: reason})
+  end
+
+  def call(conn, {:param_cast, _}) do
+    conn
+    |> put_status(:bad_request)
+    |> json(dgettext("errors", "Invalid parameters"))
+  end
+
+  def call(conn, _) do
+    conn
+    |> put_status(:internal_server_error)
+    |> json(%{error: dgettext("errors", "Something went wrong")})
+  end
+end
--- a/lib/pleroma/web/admin_api/controllers/status_controller.ex
+++ b/lib/pleroma/web/admin_api/controllers/status_controller.ex
@ -0,0 +1,79 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.AdminAPI.StatusController do
+  use Pleroma.Web, :controller
+
+  alias Pleroma.Activity
+  alias Pleroma.ModerationLog
+  alias Pleroma.Plugs.OAuthScopesPlug
+  alias Pleroma.Web.ActivityPub.ActivityPub
+  alias Pleroma.Web.CommonAPI
+  alias Pleroma.Web.MastodonAPI
+
+  require Logger
+
+  plug(Pleroma.Web.ApiSpec.CastAndValidate)
+  plug(OAuthScopesPlug, %{scopes: ["read:statuses"], admin: true} when action in [:index, :show])
+
+  plug(
+    OAuthScopesPlug,
+    %{scopes: ["write:statuses"], admin: true} when action in [:update, :delete]
+  )
+
+  action_fallback(Pleroma.Web.AdminAPI.FallbackController)
+
+  defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.Admin.StatusOperation
+
+  def index(%{assigns: %{user: _admin}} = conn, params) do
+    activities =
+      ActivityPub.fetch_statuses(nil, %{
+        "godmode" => params.godmode,
+        "local_only" => params.local_only,
+        "limit" => params.page_size,
+        "offset" => (params.page - 1) * params.page_size,
+        "exclude_reblogs" => not params.with_reblogs
+      })
+
+    render(conn, "index.json", activities: activities, as: :activity)
+  end
+
+  def show(conn, %{id: id}) do
+    with %Activity{} = activity <- Activity.get_by_id(id) do
+      conn
+      |> put_view(MastodonAPI.StatusView)
+      |> render("show.json", %{activity: activity})
+    else
+      nil -> {:error, :not_found}
+    end
+  end
+
+  def update(%{assigns: %{user: admin}, body_params: params} = conn, %{id: id}) do
+    with {:ok, activity} <- CommonAPI.update_activity_scope(id, params) do
+      ModerationLog.insert_log(%{
+        action: "status_update",
+        actor: admin,
+        subject: activity,
+        sensitive: params[:sensitive],
+        visibility: params[:visibility]
+      })
+
+      conn
+      |> put_view(MastodonAPI.StatusView)
+      |> render("show.json", %{activity: activity})
+    end
+  end
+
+  def delete(%{assigns: %{user: user}} = conn, %{id: id}) do
+    with {:ok, %Activity{}} <- CommonAPI.delete(id, user) do
+      ModerationLog.insert_log(%{
+        action: "status_delete",
+        actor: user,
+        subject_id: id
+      })
+
+      json(conn, %{})
+    end
+  end
+end
--- a/lib/pleroma/web/admin_api/search.ex
+++ b/lib/pleroma/web/admin_api/search.ex
@ -21,6 +21,7 @@ def user(params \\ %{}) do
    query =
      params
      |> Map.drop([:page, :page_size])
+      |> Map.put(:exclude_service_users, true)
      |> User.Query.build()
      |> order_by([u], u.nickname)

--- a/lib/pleroma/web/admin_api/views/account_view.ex
+++ b/lib/pleroma/web/admin_api/views/account_view.ex
@ -6,7 +6,9 @@ defmodule Pleroma.Web.AdminAPI.AccountView do
  use Pleroma.Web, :view

  alias Pleroma.User
+  alias Pleroma.Web.AdminAPI
  alias Pleroma.Web.AdminAPI.AccountView
+  alias Pleroma.Web.MastodonAPI
  alias Pleroma.Web.MediaProxy

  def render("index.json", %{users: users, count: count, page_size: page_size}) do
@ -119,6 +121,13 @@ def render("create-error.json", %{changeset: %Ecto.Changeset{changes: changes, e
    }
  end

+  def merge_account_views(%User{} = user) do
+    MastodonAPI.AccountView.render("show.json", %{user: user})
+    |> Map.merge(AdminAPI.AccountView.render("show.json", %{user: user}))
+  end
+
+  def merge_account_views(_), do: %{}
+
  defp parse_error([]), do: ""

  defp parse_error(errors) do
--- a/lib/pleroma/web/admin_api/views/report_view.ex
+++ b/lib/pleroma/web/admin_api/views/report_view.ex
@ -7,10 +7,13 @@ defmodule Pleroma.Web.AdminAPI.ReportView do

  alias Pleroma.HTML
  alias Pleroma.User
+  alias Pleroma.Web.AdminAPI
  alias Pleroma.Web.AdminAPI.Report
  alias Pleroma.Web.CommonAPI.Utils
  alias Pleroma.Web.MastodonAPI.StatusView

+  defdelegate merge_account_views(user), to: AdminAPI.AccountView
+
  def render("index.json", %{reports: reports}) do
    %{
      reports:
@ -41,8 +44,7 @@ def render("show.json", %{report: report, user: user, account: account, statuses
      statuses:
        StatusView.render("index.json", %{
          activities: statuses,
-          as: :activity,
-          skip_relationships: false
+          as: :activity
        }),
      state: report.data["state"],
      notes: render(__MODULE__, "index_notes.json", %{notes: report.report_notes})
@ -70,11 +72,4 @@ def render("show_note.json", %{
      created_at: Utils.to_masto_date(inserted_at)
    }
  end
-
-  defp merge_account_views(%User{} = user) do
-    Pleroma.Web.MastodonAPI.AccountView.render("show.json", %{user: user})
-    |> Map.merge(Pleroma.Web.AdminAPI.AccountView.render("show.json", %{user: user}))
-  end
-
-  defp merge_account_views(_), do: %{}
 end
--- a/lib/pleroma/web/admin_api/views/status_view.ex
+++ b/lib/pleroma/web/admin_api/views/status_view.ex
@ -7,24 +7,19 @@ defmodule Pleroma.Web.AdminAPI.StatusView do

  require Pleroma.Constants

-  alias Pleroma.User
-  alias Pleroma.Web.MastodonAPI.StatusView
+  alias Pleroma.Web.AdminAPI
+  alias Pleroma.Web.MastodonAPI
+
+  defdelegate merge_account_views(user), to: AdminAPI.AccountView

  def render("index.json", opts) do
    safe_render_many(opts.activities, __MODULE__, "show.json", opts)
  end

  def render("show.json", %{activity: %{data: %{"object" => _object}} = activity} = opts) do
-    user = StatusView.get_user(activity.data["actor"])
+    user = MastodonAPI.StatusView.get_user(activity.data["actor"])

-    StatusView.render("show.json", opts)
+    MastodonAPI.StatusView.render("show.json", opts)
    |> Map.merge(%{account: merge_account_views(user)})
  end
-
-  defp merge_account_views(%User{} = user) do
-    Pleroma.Web.MastodonAPI.AccountView.render("show.json", %{user: user})
-    |> Map.merge(Pleroma.Web.AdminAPI.AccountView.render("show.json", %{user: user}))
-  end
-
-  defp merge_account_views(_), do: %{}
 end
--- a/lib/pleroma/web/api_spec.ex
+++ b/lib/pleroma/web/api_spec.ex
@ -39,7 +39,12 @@ def spec do
              password: %OpenApiSpex.OAuthFlow{
                authorizationUrl: "/oauth/authorize",
                tokenUrl: "/oauth/token",
-                scopes: %{"read" => "read", "write" => "write", "follow" => "follow"}
+                scopes: %{
+                  "read" => "read",
+                  "write" => "write",
+                  "follow" => "follow",
+                  "push" => "push"
+                }
              }
            }
          }
--- a/lib/pleroma/web/api_spec/cast_and_validate.ex
+++ b/lib/pleroma/web/api_spec/cast_and_validate.ex
@ -0,0 +1,139 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2019-2020 Moxley Stratton, Mike Buhot <https://github.com/open-api-spex/open_api_spex>, MPL-2.0
+# Copyright © 2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.CastAndValidate do
+  @moduledoc """
+  This plug is based on [`OpenApiSpex.Plug.CastAndValidate`]
+  (https://github.com/open-api-spex/open_api_spex/blob/master/lib/open_api_spex/plug/cast_and_validate.ex).
+  The main difference is ignoring unexpected query params instead of throwing
+  an error and a config option (`[Pleroma.Web.ApiSpec.CastAndValidate, :strict]`)
+  to disable this behavior. Also, the default rendering error module
+  is `Pleroma.Web.ApiSpec.RenderError`.
+  """
+
+  @behaviour Plug
+
+  alias Plug.Conn
+
+  @impl Plug
+  def init(opts) do
+    opts
+    |> Map.new()
+    |> Map.put_new(:render_error, Pleroma.Web.ApiSpec.RenderError)
+  end
+
+  @impl Plug
+  def call(%{private: %{open_api_spex: private_data}} = conn, %{
+        operation_id: operation_id,
+        render_error: render_error
+      }) do
+    spec = private_data.spec
+    operation = private_data.operation_lookup[operation_id]
+
+    content_type =
+      case Conn.get_req_header(conn, "content-type") do
+        [header_value | _] ->
+          header_value
+          |> String.split(";")
+          |> List.first()
+
+        _ ->
+          nil
+      end
+
+    private_data = Map.put(private_data, :operation_id, operation_id)
+    conn = Conn.put_private(conn, :open_api_spex, private_data)
+
+    case cast_and_validate(spec, operation, conn, content_type, strict?()) do
+      {:ok, conn} ->
+        conn
+
+      {:error, reason} ->
+        opts = render_error.init(reason)
+
+        conn
+        |> render_error.call(opts)
+        |> Plug.Conn.halt()
+    end
+  end
+
+  def call(
+        %{
+          private: %{
+            phoenix_controller: controller,
+            phoenix_action: action,
+            open_api_spex: private_data
+          }
+        } = conn,
+        opts
+      ) do
+    operation =
+      case private_data.operation_lookup[{controller, action}] do
+        nil ->
+          operation_id = controller.open_api_operation(action).operationId
+          operation = private_data.operation_lookup[operation_id]
+
+          operation_lookup =
+            private_data.operation_lookup
+            |> Map.put({controller, action}, operation)
+
+          OpenApiSpex.Plug.Cache.adapter().put(
+            private_data.spec_module,
+            {private_data.spec, operation_lookup}
+          )
+
+          operation
+
+        operation ->
+          operation
+      end
+
+    if operation.operationId do
+      call(conn, Map.put(opts, :operation_id, operation.operationId))
+    else
+      raise "operationId was not found in action API spec"
+    end
+  end
+
+  def call(conn, opts), do: OpenApiSpex.Plug.CastAndValidate.call(conn, opts)
+
+  defp cast_and_validate(spec, operation, conn, content_type, true = _strict) do
+    OpenApiSpex.cast_and_validate(spec, operation, conn, content_type)
+  end
+
+  defp cast_and_validate(spec, operation, conn, content_type, false = _strict) do
+    case OpenApiSpex.cast_and_validate(spec, operation, conn, content_type) do
+      {:ok, conn} ->
+        {:ok, conn}
+
+      # Remove unexpected query params and cast/validate again
+      {:error, errors} ->
+        query_params =
+          Enum.reduce(errors, conn.query_params, fn
+            %{reason: :unexpected_field, name: name, path: [name]}, params ->
+              Map.delete(params, name)
+
+            %{reason: :invalid_enum, name: nil, path: path, value: value}, params ->
+              path = path |> Enum.reverse() |> tl() |> Enum.reverse() |> list_items_to_string()
+              update_in(params, path, &List.delete(&1, value))
+
+            _, params ->
+              params
+          end)
+
+        conn = %Conn{conn | query_params: query_params}
+        OpenApiSpex.cast_and_validate(spec, operation, conn, content_type)
+    end
+  end
+
+  defp list_items_to_string(list) do
+    Enum.map(list, fn
+      i when is_atom(i) -> to_string(i)
+      i -> i
+    end)
+  end
+
+  defp strict?, do: Pleroma.Config.get([__MODULE__, :strict], false)
+end
--- a/lib/pleroma/web/api_spec/helpers.ex
+++ b/lib/pleroma/web/api_spec/helpers.ex
@ -5,6 +5,7 @@
 defmodule Pleroma.Web.ApiSpec.Helpers do
  alias OpenApiSpex.Operation
  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Schemas.BooleanLike

  def request_body(description, schema_ref, opts \\ []) do
    media_types = ["application/json", "multipart/form-data", "application/x-www-form-urlencoded"]
@ -47,6 +48,15 @@ def pagination_params do
    ]
  end

+  def with_relationships_param do
+    Operation.parameter(
+      :with_relationships,
+      :query,
+      BooleanLike,
+      "Embed relationships into accounts."
+    )
+  end
+
  def empty_object_response do
    Operation.response("Empty object", "application/json", %Schema{type: :object, example: %{}})
  end
@ -54,4 +64,8 @@ def empty_object_response do
  def empty_array_response do
    Operation.response("Empty array", "application/json", %Schema{type: :array, example: []})
  end
+
+  def no_content_response do
+    Operation.response("No Content", "application/json", %Schema{type: :string, example: ""})
+  end
 end
--- a/lib/pleroma/web/api_spec/operations/account_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/account_operation.ex
@ -11,6 +11,7 @@ defmodule Pleroma.Web.ApiSpec.AccountOperation do
  alias Pleroma.Web.ApiSpec.Schemas.ActorType
  alias Pleroma.Web.ApiSpec.Schemas.ApiError
  alias Pleroma.Web.ApiSpec.Schemas.BooleanLike
+  alias Pleroma.Web.ApiSpec.Schemas.List
  alias Pleroma.Web.ApiSpec.Schemas.Status
  alias Pleroma.Web.ApiSpec.Schemas.VisibilityScope

@ -154,8 +155,10 @@ def followers_operation do
      security: [%{"oAuth" => ["read:accounts"]}],
      description:
        "Accounts which follow the given account, if network is not hidden by the account owner.",
-      parameters:
-        [%Reference{"$ref": "#/components/parameters/accountIdOrNickname"}] ++ pagination_params(),
+      parameters: [
+        %Reference{"$ref": "#/components/parameters/accountIdOrNickname"},
+        with_relationships_param() | pagination_params()
+      ],
      responses: %{
        200 => Operation.response("Accounts", "application/json", array_of_accounts())
      }
@ -170,8 +173,10 @@ def following_operation do
      security: [%{"oAuth" => ["read:accounts"]}],
      description:
        "Accounts which the given account is following, if network is not hidden by the account owner.",
-      parameters:
-        [%Reference{"$ref": "#/components/parameters/accountIdOrNickname"}] ++ pagination_params(),
+      parameters: [
+        %Reference{"$ref": "#/components/parameters/accountIdOrNickname"},
+        with_relationships_param() | pagination_params()
+      ],
      responses: %{200 => Operation.response("Accounts", "application/json", array_of_accounts())}
    }
  end
@ -366,15 +371,18 @@ defp create_request do
      title: "AccountCreateRequest",
      description: "POST body for creating an account",
      type: :object,
+      required: [:username, :password, :agreement],
      properties: %{
        reason: %Schema{
          type: :string,
+          nullable: true,
          description:
            "Text that will be reviewed by moderators if registrations require manual approval"
        },
        username: %Schema{type: :string, description: "The desired username for the account"},
        email: %Schema{
          type: :string,
+          nullable: true,
          description:
            "The email address to be used for login. Required when `account_activation_required` is enabled.",
          format: :email
@ -385,29 +393,39 @@ defp create_request do
          format: :password
        },
        agreement: %Schema{
-          type: :boolean,
+          allOf: [BooleanLike],
          description:
            "Whether the user agrees to the local rules, terms, and policies. These should be presented to the user in order to allow them to consent before setting this parameter to TRUE."
        },
        locale: %Schema{
          type: :string,
+          nullable: true,
          description: "The language of the confirmation email that will be sent"
        },
        # Pleroma-specific properties:
-        fullname: %Schema{type: :string, description: "Full name"},
-        bio: %Schema{type: :string, description: "Bio", default: ""},
+        fullname: %Schema{type: :string, nullable: true, description: "Full name"},
+        bio: %Schema{type: :string, description: "Bio", nullable: true, default: ""},
        captcha_solution: %Schema{
          type: :string,
+          nullable: true,
          description: "Provider-specific captcha solution"
        },
-        captcha_token: %Schema{type: :string, description: "Provider-specific captcha token"},
-        captcha_answer_data: %Schema{type: :string, description: "Provider-specific captcha data"},
+        captcha_token: %Schema{
+          type: :string,
+          nullable: true,
+          description: "Provider-specific captcha token"
+        },
+        captcha_answer_data: %Schema{
+          type: :string,
+          nullable: true,
+          description: "Provider-specific captcha data"
+        },
        token: %Schema{
          type: :string,
+          nullable: true,
          description: "Invite token required when the registrations aren't public"
        }
      },
-      required: [:username, :password, :agreement],
      example: %{
        "username" => "cofe",
        "email" => "cofe@example.com",
@ -445,29 +463,35 @@ defp update_creadentials_request do
      type: :object,
      properties: %{
        bot: %Schema{
-          type: :boolean,
+          allOf: [BooleanLike],
+          nullable: true,
          description: "Whether the account has a bot flag."
        },
        display_name: %Schema{
          type: :string,
+          nullable: true,
          description: "The display name to use for the profile."
        },
        note: %Schema{type: :string, description: "The account bio."},
        avatar: %Schema{
          type: :string,
+          nullable: true,
          description: "Avatar image encoded using multipart/form-data",
          format: :binary
        },
        header: %Schema{
          type: :string,
+          nullable: true,
          description: "Header image encoded using multipart/form-data",
          format: :binary
        },
        locked: %Schema{
-          type: :boolean,
+          allOf: [BooleanLike],
+          nullable: true,
          description: "Whether manual approval of follow requests is required."
        },
        fields_attributes: %Schema{
+          nullable: true,
          oneOf: [
            %Schema{type: :array, items: attribute_field()},
            %Schema{type: :object, additionalProperties: %Schema{type: attribute_field()}}
@ -486,48 +510,66 @@ defp update_creadentials_request do

        # Pleroma-specific fields
        no_rich_text: %Schema{
-          type: :boolean,
+          allOf: [BooleanLike],
+          nullable: true,
          description: "html tags are stripped from all statuses requested from the API"
        },
-        hide_followers: %Schema{type: :boolean, description: "user's followers will be hidden"},
-        hide_follows: %Schema{type: :boolean, description: "user's follows will be hidden"},
+        hide_followers: %Schema{
+          allOf: [BooleanLike],
+          nullable: true,
+          description: "user's followers will be hidden"
+        },
+        hide_follows: %Schema{
+          allOf: [BooleanLike],
+          nullable: true,
+          description: "user's follows will be hidden"
+        },
        hide_followers_count: %Schema{
-          type: :boolean,
+          allOf: [BooleanLike],
+          nullable: true,
          description: "user's follower count will be hidden"
        },
        hide_follows_count: %Schema{
-          type: :boolean,
+          allOf: [BooleanLike],
+          nullable: true,
          description: "user's follow count will be hidden"
        },
        hide_favorites: %Schema{
-          type: :boolean,
+          allOf: [BooleanLike],
+          nullable: true,
          description: "user's favorites timeline will be hidden"
        },
        show_role: %Schema{
-          type: :boolean,
+          allOf: [BooleanLike],
+          nullable: true,
          description: "user's role (e.g admin, moderator) will be exposed to anyone in the
        API"
        },
        default_scope: VisibilityScope,
        pleroma_settings_store: %Schema{
          type: :object,
+          nullable: true,
          description: "Opaque user settings to be saved on the backend."
        },
        skip_thread_containment: %Schema{
-          type: :boolean,
+          allOf: [BooleanLike],
+          nullable: true,
          description: "Skip filtering out broken threads"
        },
        allow_following_move: %Schema{
-          type: :boolean,
+          allOf: [BooleanLike],
+          nullable: true,
          description: "Allows automatically follow moved following accounts"
        },
        pleroma_background_image: %Schema{
          type: :string,
+          nullable: true,
          description: "Sets the background image of the user.",
          format: :binary
        },
        discoverable: %Schema{
-          type: :boolean,
+          allOf: [BooleanLike],
+          nullable: true,
          description:
            "Discovery of this account in search results and other services is allowed."
        },
@ -555,11 +597,12 @@ defp update_creadentials_request do
    }
  end

-  defp array_of_accounts do
+  def array_of_accounts do
    %Schema{
      title: "ArrayOfAccounts",
      type: :array,
-      items: Account
+      items: Account,
+      example: [Account.schema().example]
    }
  end

@ -622,7 +665,7 @@ defp follow_by_uri_request do
      description: "POST body for muting an account",
      type: :object,
      properties: %{
-        uri: %Schema{type: :string, format: :uri}
+        uri: %Schema{type: :string, nullable: true, format: :uri}
      },
      required: [:uri]
    }
@ -635,7 +678,8 @@ defp mute_request do
      type: :object,
      properties: %{
        notifications: %Schema{
-          type: :boolean,
+          allOf: [BooleanLike],
+          nullable: true,
          description: "Mute notifications in addition to statuses? Defaults to true.",
          default: true
        }
@ -646,28 +690,12 @@ defp mute_request do
    }
  end

-  defp list do
-    %Schema{
-      title: "List",
-      description: "Response schema for a list",
-      type: :object,
-      properties: %{
-        id: %Schema{type: :string},
-        title: %Schema{type: :string}
-      },
-      example: %{
-        "id" => "123",
-        "title" => "my list"
-      }
-    }
-  end
-
  defp array_of_lists do
    %Schema{
      title: "ArrayOfLists",
      description: "Response schema for lists",
      type: :array,
-      items: list(),
+      items: List,
      example: [
        %{"id" => "123", "title" => "my list"},
        %{"id" => "1337", "title" => "anotehr list"}
--- a/lib/pleroma/web/api_spec/operations/admin/status_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/admin/status_operation.ex
@ -0,0 +1,165 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Admin.StatusOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Schemas.Account
+  alias Pleroma.Web.ApiSpec.Schemas.ApiError
+  alias Pleroma.Web.ApiSpec.Schemas.FlakeID
+  alias Pleroma.Web.ApiSpec.Schemas.Status
+  alias Pleroma.Web.ApiSpec.Schemas.VisibilityScope
+
+  import Pleroma.Web.ApiSpec.Helpers
+  import Pleroma.Web.ApiSpec.StatusOperation, only: [id_param: 0]
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def index_operation do
+    %Operation{
+      tags: ["Admin", "Statuses"],
+      operationId: "AdminAPI.StatusController.index",
+      security: [%{"oAuth" => ["read:statuses"]}],
+      parameters: [
+        Operation.parameter(
+          :godmode,
+          :query,
+          %Schema{type: :boolean, default: false},
+          "Allows to see private statuses"
+        ),
+        Operation.parameter(
+          :local_only,
+          :query,
+          %Schema{type: :boolean, default: false},
+          "Excludes remote statuses"
+        ),
+        Operation.parameter(
+          :with_reblogs,
+          :query,
+          %Schema{type: :boolean, default: false},
+          "Allows to see reblogs"
+        ),
+        Operation.parameter(
+          :page,
+          :query,
+          %Schema{type: :integer, default: 1},
+          "Page"
+        ),
+        Operation.parameter(
+          :page_size,
+          :query,
+          %Schema{type: :integer, default: 50},
+          "Number of statuses to return"
+        )
+      ],
+      responses: %{
+        200 =>
+          Operation.response("Array of statuses", "application/json", %Schema{
+            type: :array,
+            items: status()
+          })
+      }
+    }
+  end
+
+  def show_operation do
+    %Operation{
+      tags: ["Admin", "Statuses"],
+      summary: "Show Status",
+      operationId: "AdminAPI.StatusController.show",
+      parameters: [id_param()],
+      security: [%{"oAuth" => ["read:statuses"]}],
+      responses: %{
+        200 => Operation.response("Status", "application/json", Status),
+        404 => Operation.response("Not Found", "application/json", ApiError)
+      }
+    }
+  end
+
+  def update_operation do
+    %Operation{
+      tags: ["Admin", "Statuses"],
+      summary: "Change the scope of an individual reported status",
+      operationId: "AdminAPI.StatusController.update",
+      parameters: [id_param()],
+      security: [%{"oAuth" => ["write:statuses"]}],
+      requestBody: request_body("Parameters", update_request(), required: true),
+      responses: %{
+        200 => Operation.response("Status", "application/json", Status),
+        400 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  def delete_operation do
+    %Operation{
+      tags: ["Admin", "Statuses"],
+      summary: "Delete an individual reported status",
+      operationId: "AdminAPI.StatusController.delete",
+      parameters: [id_param()],
+      security: [%{"oAuth" => ["write:statuses"]}],
+      responses: %{
+        200 => empty_object_response(),
+        404 => Operation.response("Not Found", "application/json", ApiError)
+      }
+    }
+  end
+
+  defp status do
+    %Schema{
+      anyOf: [
+        Status,
+        %Schema{
+          type: :object,
+          properties: %{
+            account: %Schema{allOf: [Account, admin_account()]}
+          }
+        }
+      ]
+    }
+  end
+
+  defp admin_account do
+    %Schema{
+      type: :object,
+      properties: %{
+        id: FlakeID,
+        avatar: %Schema{type: :string},
+        nickname: %Schema{type: :string},
+        display_name: %Schema{type: :string},
+        deactivated: %Schema{type: :boolean},
+        local: %Schema{type: :boolean},
+        roles: %Schema{
+          type: :object,
+          properties: %{
+            admin: %Schema{type: :boolean},
+            moderator: %Schema{type: :boolean}
+          }
+        },
+        tags: %Schema{type: :string},
+        confirmation_pending: %Schema{type: :string}
+      }
+    }
+  end
+
+  defp update_request do
+    %Schema{
+      type: :object,
+      properties: %{
+        sensitive: %Schema{
+          type: :boolean,
+          description: "Mark status and attached media as sensitive?"
+        },
+        visibility: VisibilityScope
+      },
+      example: %{
+        "visibility" => "private",
+        "sensitive" => "false"
+      }
+    }
+  end
+end
--- a/lib/pleroma/web/api_spec/operations/app_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/app_operation.ex
@ -105,7 +105,11 @@ defp create_request do
          description: "Space separated list of scopes",
          default: "read"
        },
-        website: %Schema{type: :string, description: "A URL to the homepage of your app"}
+        website: %Schema{
+          type: :string,
+          nullable: true,
+          description: "A URL to the homepage of your app"
+        }
      },
      required: [:client_name, :redirect_uris],
      example: %{
--- a/lib/pleroma/web/api_spec/operations/conversation_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/conversation_operation.ex
@ -0,0 +1,61 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.ConversationOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Schemas.Conversation
+  alias Pleroma.Web.ApiSpec.Schemas.FlakeID
+
+  import Pleroma.Web.ApiSpec.Helpers
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def index_operation do
+    %Operation{
+      tags: ["Conversations"],
+      summary: "Show conversation",
+      security: [%{"oAuth" => ["read:statuses"]}],
+      operationId: "ConversationController.index",
+      parameters: [
+        Operation.parameter(
+          :recipients,
+          :query,
+          %Schema{type: :array, items: FlakeID},
+          "Only return conversations with the given recipients (a list of user ids)"
+        )
+        | pagination_params()
+      ],
+      responses: %{
+        200 =>
+          Operation.response("Array of Conversation", "application/json", %Schema{
+            type: :array,
+            items: Conversation,
+            example: [Conversation.schema().example]
+          })
+      }
+    }
+  end
+
+  def mark_as_read_operation do
+    %Operation{
+      tags: ["Conversations"],
+      summary: "Mark as read",
+      operationId: "ConversationController.mark_as_read",
+      parameters: [
+        Operation.parameter(:id, :path, :string, "Conversation ID",
+          example: "123",
+          required: true
+        )
+      ],
+      security: [%{"oAuth" => ["write:conversations"]}],
+      responses: %{
+        200 => Operation.response("Conversation", "application/json", Conversation)
+      }
+    }
+  end
+end
--- a/lib/pleroma/web/api_spec/operations/emoji_reaction_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/emoji_reaction_operation.ex
@ -0,0 +1,104 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.EmojiReactionOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Schemas.Account
+  alias Pleroma.Web.ApiSpec.Schemas.ApiError
+  alias Pleroma.Web.ApiSpec.Schemas.FlakeID
+  alias Pleroma.Web.ApiSpec.Schemas.Status
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def index_operation do
+    %Operation{
+      tags: ["Emoji Reactions"],
+      summary:
+        "Get an object of emoji to account mappings with accounts that reacted to the post",
+      parameters: [
+        Operation.parameter(:id, :path, FlakeID, "Status ID", required: true),
+        Operation.parameter(:emoji, :path, :string, "Filter by a single unicode emoji",
+          required: false
+        )
+      ],
+      security: [%{"oAuth" => ["read:statuses"]}],
+      operationId: "EmojiReactionController.index",
+      responses: %{
+        200 => array_of_reactions_response()
+      }
+    }
+  end
+
+  def create_operation do
+    %Operation{
+      tags: ["Emoji Reactions"],
+      summary: "React to a post with a unicode emoji",
+      parameters: [
+        Operation.parameter(:id, :path, FlakeID, "Status ID", required: true),
+        Operation.parameter(:emoji, :path, :string, "A single character unicode emoji",
+          required: true
+        )
+      ],
+      security: [%{"oAuth" => ["write:statuses"]}],
+      operationId: "EmojiReactionController.create",
+      responses: %{
+        200 => Operation.response("Status", "application/json", Status),
+        400 => Operation.response("Bad Request", "application/json", ApiError)
+      }
+    }
+  end
+
+  def delete_operation do
+    %Operation{
+      tags: ["Emoji Reactions"],
+      summary: "Remove a reaction to a post with a unicode emoji",
+      parameters: [
+        Operation.parameter(:id, :path, FlakeID, "Status ID", required: true),
+        Operation.parameter(:emoji, :path, :string, "A single character unicode emoji",
+          required: true
+        )
+      ],
+      security: [%{"oAuth" => ["write:statuses"]}],
+      operationId: "EmojiReactionController.delete",
+      responses: %{
+        200 => Operation.response("Status", "application/json", Status)
+      }
+    }
+  end
+
+  defp array_of_reactions_response do
+    Operation.response("Array of Emoji Reactions", "application/json", %Schema{
+      type: :array,
+      items: emoji_reaction(),
+      example: [emoji_reaction().example]
+    })
+  end
+
+  defp emoji_reaction do
+    %Schema{
+      title: "EmojiReaction",
+      type: :object,
+      properties: %{
+        name: %Schema{type: :string, description: "Emoji"},
+        count: %Schema{type: :integer, description: "Count of reactions with this emoji"},
+        me: %Schema{type: :boolean, description: "Did I react with this emoji?"},
+        accounts: %Schema{
+          type: :array,
+          items: Account,
+          description: "Array of accounts reacted with this emoji"
+        }
+      },
+      example: %{
+        "name" => "😱",
+        "count" => 1,
+        "me" => false,
+        "accounts" => [Account.schema().example]
+      }
+    }
+  end
+end
--- a/lib/pleroma/web/api_spec/operations/filter_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/filter_operation.ex
@ -0,0 +1,230 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.FilterOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Helpers
+  alias Pleroma.Web.ApiSpec.Schemas.BooleanLike
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def index_operation do
+    %Operation{
+      tags: ["apps"],
+      summary: "View all filters",
+      operationId: "FilterController.index",
+      security: [%{"oAuth" => ["read:filters"]}],
+      responses: %{
+        200 => Operation.response("Filters", "application/json", array_of_filters())
+      }
+    }
+  end
+
+  def create_operation do
+    %Operation{
+      tags: ["apps"],
+      summary: "Create a filter",
+      operationId: "FilterController.create",
+      requestBody: Helpers.request_body("Parameters", create_request(), required: true),
+      security: [%{"oAuth" => ["write:filters"]}],
+      responses: %{200 => Operation.response("Filter", "application/json", filter())}
+    }
+  end
+
+  def show_operation do
+    %Operation{
+      tags: ["apps"],
+      summary: "View all filters",
+      parameters: [id_param()],
+      operationId: "FilterController.show",
+      security: [%{"oAuth" => ["read:filters"]}],
+      responses: %{
+        200 => Operation.response("Filter", "application/json", filter())
+      }
+    }
+  end
+
+  def update_operation do
+    %Operation{
+      tags: ["apps"],
+      summary: "Update a filter",
+      parameters: [id_param()],
+      operationId: "FilterController.update",
+      requestBody: Helpers.request_body("Parameters", update_request(), required: true),
+      security: [%{"oAuth" => ["write:filters"]}],
+      responses: %{
+        200 => Operation.response("Filter", "application/json", filter())
+      }
+    }
+  end
+
+  def delete_operation do
+    %Operation{
+      tags: ["apps"],
+      summary: "Remove a filter",
+      parameters: [id_param()],
+      operationId: "FilterController.delete",
+      security: [%{"oAuth" => ["write:filters"]}],
+      responses: %{
+        200 =>
+          Operation.response("Filter", "application/json", %Schema{
+            type: :object,
+            description: "Empty object"
+          })
+      }
+    }
+  end
+
+  defp id_param do
+    Operation.parameter(:id, :path, :string, "Filter ID", example: "123", required: true)
+  end
+
+  defp filter do
+    %Schema{
+      title: "Filter",
+      type: :object,
+      properties: %{
+        id: %Schema{type: :string},
+        phrase: %Schema{type: :string, description: "The text to be filtered"},
+        context: %Schema{
+          type: :array,
+          items: %Schema{type: :string, enum: ["home", "notifications", "public", "thread"]},
+          description: "The contexts in which the filter should be applied."
+        },
+        expires_at: %Schema{
+          type: :string,
+          format: :"date-time",
+          description:
+            "When the filter should no longer be applied. String (ISO 8601 Datetime), or null if the filter does not expire.",
+          nullable: true
+        },
+        irreversible: %Schema{
+          type: :boolean,
+          description:
+            "Should matching entities in home and notifications be dropped by the server?"
+        },
+        whole_word: %Schema{
+          type: :boolean,
+          description: "Should the filter consider word boundaries?"
+        }
+      },
+      example: %{
+        "id" => "5580",
+        "phrase" => "@twitter.com",
+        "context" => [
+          "home",
+          "notifications",
+          "public",
+          "thread"
+        ],
+        "whole_word" => false,
+        "expires_at" => nil,
+        "irreversible" => true
+      }
+    }
+  end
+
+  defp array_of_filters do
+    %Schema{
+      title: "ArrayOfFilters",
+      description: "Array of Filters",
+      type: :array,
+      items: filter(),
+      example: [
+        %{
+          "id" => "5580",
+          "phrase" => "@twitter.com",
+          "context" => [
+            "home",
+            "notifications",
+            "public",
+            "thread"
+          ],
+          "whole_word" => false,
+          "expires_at" => nil,
+          "irreversible" => true
+        },
+        %{
+          "id" => "6191",
+          "phrase" => ":eurovision2019:",
+          "context" => [
+            "home"
+          ],
+          "whole_word" => true,
+          "expires_at" => "2019-05-21T13:47:31.333Z",
+          "irreversible" => false
+        }
+      ]
+    }
+  end
+
+  defp create_request do
+    %Schema{
+      title: "FilterCreateRequest",
+      allOf: [
+        update_request(),
+        %Schema{
+          type: :object,
+          properties: %{
+            irreversible: %Schema{
+              allOf: [BooleanLike],
+              description:
+                "Should the server irreversibly drop matching entities from home and notifications?",
+              default: false
+            }
+          }
+        }
+      ],
+      example: %{
+        "phrase" => "knights",
+        "context" => ["home"]
+      }
+    }
+  end
+
+  defp update_request do
+    %Schema{
+      title: "FilterUpdateRequest",
+      type: :object,
+      properties: %{
+        phrase: %Schema{type: :string, description: "The text to be filtered"},
+        context: %Schema{
+          type: :array,
+          items: %Schema{type: :string, enum: ["home", "notifications", "public", "thread"]},
+          description:
+            "Array of enumerable strings `home`, `notifications`, `public`, `thread`. At least one context must be specified."
+        },
+        irreversible: %Schema{
+          allOf: [BooleanLike],
+          nullable: true,
+          description:
+            "Should the server irreversibly drop matching entities from home and notifications?"
+        },
+        whole_word: %Schema{
+          allOf: [BooleanLike],
+          nullable: true,
+          description: "Consider word boundaries?",
+          default: true
+        }
+        # TODO: probably should implement filter expiration
+        # expires_in: %Schema{
+        #   type: :string,
+        #   format: :"date-time",
+        #   description:
+        #     "ISO 8601 Datetime for when the filter expires. Otherwise,
+        #  null for a filter that doesn't expire."
+        # }
+      },
+      required: [:phrase, :context],
+      example: %{
+        "phrase" => "knights",
+        "context" => ["home"]
+      }
+    }
+  end
+end
--- a/lib/pleroma/web/api_spec/operations/follow_request_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/follow_request_operation.ex
@ -0,0 +1,65 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.FollowRequestOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Schemas.Account
+  alias Pleroma.Web.ApiSpec.Schemas.AccountRelationship
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def index_operation do
+    %Operation{
+      tags: ["Follow Requests"],
+      summary: "Pending Follows",
+      security: [%{"oAuth" => ["read:follows", "follow"]}],
+      operationId: "FollowRequestController.index",
+      responses: %{
+        200 =>
+          Operation.response("Array of Account", "application/json", %Schema{
+            type: :array,
+            items: Account,
+            example: [Account.schema().example]
+          })
+      }
+    }
+  end
+
+  def authorize_operation do
+    %Operation{
+      tags: ["Follow Requests"],
+      summary: "Accept Follow",
+      operationId: "FollowRequestController.authorize",
+      parameters: [id_param()],
+      security: [%{"oAuth" => ["follow", "write:follows"]}],
+      responses: %{
+        200 => Operation.response("Relationship", "application/json", AccountRelationship)
+      }
+    }
+  end
+
+  def reject_operation do
+    %Operation{
+      tags: ["Follow Requests"],
+      summary: "Reject Follow",
+      operationId: "FollowRequestController.reject",
+      parameters: [id_param()],
+      security: [%{"oAuth" => ["follow", "write:follows"]}],
+      responses: %{
+        200 => Operation.response("Relationship", "application/json", AccountRelationship)
+      }
+    }
+  end
+
+  defp id_param do
+    Operation.parameter(:id, :path, :string, "Conversation ID",
+      example: "123",
+      required: true
+    )
+  end
+end
--- a/lib/pleroma/web/api_spec/operations/instance_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/instance_operation.ex
@ -0,0 +1,175 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.InstanceOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def show_operation do
+    %Operation{
+      tags: ["Instance"],
+      summary: "Fetch instance",
+      description: "Information about the server",
+      operationId: "InstanceController.show",
+      responses: %{
+        200 => Operation.response("Instance", "application/json", instance())
+      }
+    }
+  end
+
+  def peers_operation do
+    %Operation{
+      tags: ["Instance"],
+      summary: "List of known hosts",
+      operationId: "InstanceController.peers",
+      responses: %{
+        200 => Operation.response("Array of domains", "application/json", array_of_domains())
+      }
+    }
+  end
+
+  defp instance do
+    %Schema{
+      type: :object,
+      properties: %{
+        uri: %Schema{type: :string, description: "The domain name of the instance"},
+        title: %Schema{type: :string, description: "The title of the website"},
+        description: %Schema{
+          type: :string,
+          description: "Admin-defined description of the Pleroma site"
+        },
+        version: %Schema{
+          type: :string,
+          description: "The version of Pleroma installed on the instance"
+        },
+        email: %Schema{
+          type: :string,
+          description: "An email that may be contacted for any inquiries",
+          format: :email
+        },
+        urls: %Schema{
+          type: :object,
+          description: "URLs of interest for clients apps",
+          properties: %{
+            streaming_api: %Schema{
+              type: :string,
+              description: "Websockets address for push streaming"
+            }
+          }
+        },
+        stats: %Schema{
+          type: :object,
+          description: "Statistics about how much information the instance contains",
+          properties: %{
+            user_count: %Schema{
+              type: :integer,
+              description: "Users registered on this instance"
+            },
+            status_count: %Schema{
+              type: :integer,
+              description: "Statuses authored by users on instance"
+            },
+            domain_count: %Schema{
+              type: :integer,
+              description: "Domains federated with this instance"
+            }
+          }
+        },
+        thumbnail: %Schema{
+          type: :string,
+          description: "Banner image for the website",
+          nullable: true
+        },
+        languages: %Schema{
+          type: :array,
+          items: %Schema{type: :string},
+          description: "Primary langauges of the website and its staff"
+        },
+        registrations: %Schema{type: :boolean, description: "Whether registrations are enabled"},
+        # Extra (not present in Mastodon):
+        max_toot_chars: %Schema{
+          type: :integer,
+          description: ": Posts character limit (CW/Subject included in the counter)"
+        },
+        poll_limits: %Schema{
+          type: :object,
+          description: "A map with poll limits for local polls",
+          properties: %{
+            max_options: %Schema{
+              type: :integer,
+              description: "Maximum number of options."
+            },
+            max_option_chars: %Schema{
+              type: :integer,
+              description: "Maximum number of characters per option."
+            },
+            min_expiration: %Schema{
+              type: :integer,
+              description: "Minimum expiration time (in seconds)."
+            },
+            max_expiration: %Schema{
+              type: :integer,
+              description: "Maximum expiration time (in seconds)."
+            }
+          }
+        },
+        upload_limit: %Schema{
+          type: :integer,
+          description: "File size limit of uploads (except for avatar, background, banner)"
+        },
+        avatar_upload_limit: %Schema{type: :integer, description: "The title of the website"},
+        background_upload_limit: %Schema{type: :integer, description: "The title of the website"},
+        banner_upload_limit: %Schema{type: :integer, description: "The title of the website"},
+        background_image: %Schema{
+          type: :string,
+          format: :uri,
+          description: "The background image for the website"
+        }
+      },
+      example: %{
+        "avatar_upload_limit" => 2_000_000,
+        "background_upload_limit" => 4_000_000,
+        "background_image" => "/static/image.png",
+        "banner_upload_limit" => 4_000_000,
+        "description" => "A Pleroma instance, an alternative fediverse server",
+        "email" => "lain@lain.com",
+        "languages" => ["en"],
+        "max_toot_chars" => 5000,
+        "poll_limits" => %{
+          "max_expiration" => 31_536_000,
+          "max_option_chars" => 200,
+          "max_options" => 20,
+          "min_expiration" => 0
+        },
+        "registrations" => false,
+        "stats" => %{
+          "domain_count" => 2996,
+          "status_count" => 15_802,
+          "user_count" => 5
+        },
+        "thumbnail" => "https://lain.com/instance/thumbnail.jpeg",
+        "title" => "lain.com",
+        "upload_limit" => 16_000_000,
+        "uri" => "https://lain.com",
+        "urls" => %{
+          "streaming_api" => "wss://lain.com"
+        },
+        "version" => "2.7.2 (compatible; Pleroma 2.0.50-536-g25eec6d7-develop)"
+      }
+    }
+  end
+
+  defp array_of_domains do
+    %Schema{
+      type: :array,
+      items: %Schema{type: :string},
+      example: ["pleroma.site", "lain.com", "bikeshed.party"]
+    }
+  end
+end
--- a/Show More
+++ b/Show More