Merge branch 'develop' into feature/hide-follows-remote

CHANGELOG.md

@@ -11,6 +11,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Federation: Return 403 errors when trying to request pages from a user's follower/following collections if they have `hide_followers`/`hide_follows` set
 - NodeInfo: Return `skipThreadContainment` in `metadata` for the `skip_thread_containment` option
 - Mastodon API: Unsubscribe followers when they unfollow a user
+- AdminAPI: Add "godmode" while fetching user statuses (i.e. admin can see private statuses)
 
 ### Fixed
 - Not being able to pin unlisted posts
@@ -19,13 +20,20 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Mastodon API: Handling of search timeouts (`/api/v1/search` and `/api/v2/search`)
 - Mastodon API: Embedded relationships not being properly rendered in the Account entity of Status entity
 - Mastodon API: Add `account_id`, `type`, `offset`, and `limit` to search API (`/api/v1/search` and `/api/v2/search`)
+- Mastodon API, streaming: Fix filtering of notifications based on blocks/mutes/thread mutes
 - ActivityPub C2S: follower/following collection pages being inaccessible even when authentifucated if `hide_followers`/ `hide_follows` was set
 - Existing user id not being preserved on insert conflict
+- Rich Media: Parser failing when no TTL can be found by image TTL setters
+- Rich Media: The crawled URL is now spliced into the rich media data.
+- ActivityPub S2S: sharedInbox usage has been mostly aligned with the rules in the AP specification.
 
 ### Added
 - MRF: Support for priming the mediaproxy cache (`Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy`)
 - MRF: Support for excluding specific domains from Transparency.
 - MRF: Support for filtering posts based on who they mention (`Pleroma.Web.ActivityPub.MRF.MentionPolicy`)
+- MRF (Simple Policy): Support for wildcard domains.
+- Support for wildcard domains in user domain blocks setting.
+- Configuration: `quarantined_instances` support wildcard domains.
 - Configuration: `federation_incoming_replies_max_depth` option
 - Mastodon API: Support for the [`tagged` filter](https://github.com/tootsuite/mastodon/pull/9755) in [`GET /api/v1/accounts/:id/statuses`](https://docs.joinmastodon.org/api/rest/accounts/#get-api-v1-accounts-id-statuses)
 - Mastodon API, streaming: Add support for passing the token in the `Sec-WebSocket-Protocol` header
@@ -33,6 +41,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Mastodon API: Add support for categories for custom emojis by reusing the group feature. <https://github.com/tootsuite/mastodon/pull/11196>
 - Mastodon API: Add support for muting/unmuting notifications
 - Mastodon API: Add support for the `blocked_by` attribute in the relationship API (`GET /api/v1/accounts/relationships`). <https://github.com/tootsuite/mastodon/pull/10373>
+- Mastodon API: Add support for the `domain_blocking` attribute in the relationship API (`GET /api/v1/accounts/relationships`).
 - Mastodon API: Add `pleroma.deactivated` to the Account entity
 - Mastodon API: added `/auth/password` endpoint for password reset with rate limit.
 - Mastodon API: /api/v1/accounts/:id/statuses now supports nicknames or user id
@@ -47,6 +56,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Twitter API: added rate limit for `/api/account/password_reset` endpoint.
 - ActivityPub: Add an internal service actor for fetching ActivityPub objects.
 - ActivityPub: Optional signing of ActivityPub object fetches.
+- Admin API: Endpoint for fetching latest user's statuses
 
 ### Changed
 - Configuration: Filter.AnonymizeFilename added ability to retain file extension with custom text

docs/api/admin_api.md

@@ -187,6 +187,19 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
   - On failure: `Not found`
   - On success: JSON of the user
 
+## `/api/pleroma/admin/users/:nickname_or_id/statuses`
+
+### Retrive user's latest statuses
+
+- Method: `GET`
+- Params:
+  - `nickname` or `id`
+  - *optional* `page_size`: number of statuses to return (default is `20`)
+  - *optional* `godmode`: `true`/`false` – allows to see private statuses
+- Response:
+  - On failure: `Not found`
+  - On success: JSON array of user's latest statuses
+
 ## `/api/pleroma/admin/relay`
 
 ### Follow a Relay
@@ -564,6 +577,7 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
 
 ## `/api/pleroma/admin/config`
 ### List config settings
+List config settings only works with `:pleroma => :instance => :dynamic_configuration` setting to `true`.
 - Method `GET`
 - Params: none
 - Response:
@@ -582,6 +596,7 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
 
 ## `/api/pleroma/admin/config`
 ### Update config settings
+Updating config settings only works with `:pleroma => :instance => :dynamic_configuration` setting to `true`.
 Module name can be passed as string, which starts with `Pleroma`, e.g. `"Pleroma.Upload"`.
 Atom keys and values can be passed with `:` in the beginning, e.g. `":upload"`.
 Tuples can be passed as `{"tuple": ["first_val", Pleroma.Module, []]}`.

docs/config/small_customizations.md

@@ -1,35 +1,12 @@
 # Small customizations
-Replace `dev.secret.exs` with `prod.secret.exs` according to your setup.
 
-# Thumbnail
+See also static_dir.md for visual settings.
 
-Replace `priv/static/instance/thumbnail.jpeg` with your selfie or other neat picture. It will appear in [Pleroma Instances](http://distsn.org/pleroma-instances.html).
+## Theme
-
-# Instance-specific panel
-
-![instance-specific panel demo](/uploads/296b19ec806b130e0b49b16bfe29ce8a/image.png)
-
-To show the instance specific panel, set `show_instance_panel` to `true` in `config/dev.secret.exs`. You can modify its content by editing `priv/static/instance/panel.html`.
-
-# Background
-
-You can change the background of your Pleroma instance by uploading it to `priv/static/static`, and then changing `"background"` in `config/dev.secret.exs` accordingly.
-
-# Logo
-
-![logo modification demo](/uploads/c70b14de60fa74245e7f0dcfa695ebff/image.png)
-
-If you want to give a brand to your instance, look no further. You can change the logo of your instance by uploading it to `priv/static/static`, and then changing `logo` in `config/dev.secret.exs` accordingly.
-
-# Theme
 
 All users of your instance will be able to change the theme they use by going to the settings (the cog in the top-right hand corner). However, if you wish to change the default theme, you can do so by editing `theme` in `config/dev.secret.exs` accordingly.
 
-# Terms of Service
+## Message Visibility
-
-Terms of Service will be shown to all users on the registration page. It's the best place where to write down the rules for your instance. You can modify the rules by changing `priv/static/static/terms-of-service.html`.
-
-# Message Visibility
 
 To enable message visibility options when posting like in the Mastodon frontend, set
 `scope_options_enabled` to `true` in `config/dev.secret.exs`.

docs/config/static_dir.md

@@ -7,7 +7,13 @@ config :pleroma, :instance,
   static_dir: "instance/static/",
 ```
 
-You can overwrite this value in your configuration to use a different static instance directory.
+For example, edit `instance/static/instance/panel.html` .
+
+Alternatively, you can overwrite this value in your configuration to use a different static instance directory.
+
+This document is written assuming `instance/static/`.
+
+Or, if you want to manage your custom file in git repository, basically remove the `instance/` entry from `.gitignore`.
 
 ## robots.txt
 
@@ -18,3 +24,46 @@ If you want to generate a restrictive `robots.txt`, you can run the following mi
 ```
 mix pleroma.robots_txt disallow_all
 ```
+
+## Thumbnail
+
+Put on `instance/static/instance/thumbnail.jpeg` with your selfie or other neat picture. It will appear in [Pleroma Instances](http://distsn.org/pleroma-instances.html).
+
+## Instance-specific panel
+
+![instance-specific panel demo](/uploads/296b19ec806b130e0b49b16bfe29ce8a/image.png)
+
+Create and Edit your file on `instance/static/instance/panel.html`.
+
+## Background
+
+You can change the background of your Pleroma instance by uploading it to `instance/static/`, and then changing `background` in `config/prod.secret.exs` accordingly.
+
+If you put `instance/static/images/background.jpg`
+
+```
+config :pleroma, :frontend_configurations,
+  pleroma_fe: %{
+    background: "/images/background.jpg"
+  }
+```
+
+## Logo
+
+![logo modification demo](/uploads/c70b14de60fa74245e7f0dcfa695ebff/image.png)
+
+If you want to give a brand to your instance, You can change the logo of your instance by uploading it to `instance/static/`.
+
+Alternatively, you can specify the path with config.
+If you put `instance/static/static/mylogo-file.png`
+
+```
+config :pleroma, :frontend_configurations,
+  pleroma_fe: %{
+   logo: "/static/mylogo-file.png"
+  }
+```
+
+## Terms of Service
+
+Terms of Service will be shown to all users on the registration page. It's the best place where to write down the rules for your instance. You can modify the rules by changing `instance/static/static/terms-of-service.html`.

lib/pleroma/plugs/set_format_plug.ex

@@ -0,0 +1,24 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Plugs.SetFormatPlug do
+  import Plug.Conn, only: [assign: 3, fetch_query_params: 1]
+
+  def init(_), do: nil
+
+  def call(conn, _) do
+    case get_format(conn) do
+      nil -> conn
+      format -> assign(conn, :format, format)
+    end
+  end
+
+  defp get_format(conn) do
+    conn.private[:phoenix_format] ||
+      case fetch_query_params(conn) do
+        %{query_params: %{"_format" => format}} -> format
+        _ -> nil
+      end
+  end
+end

lib/pleroma/signature.ex

@@ -10,9 +10,18 @@ defmodule Pleroma.Signature do
   alias Pleroma.Web.ActivityPub.ActivityPub
 
   def key_id_to_actor_id(key_id) do
+    uri =
       URI.parse(key_id)
       |> Map.put(:fragment, nil)
-    |> URI.to_string()
+
+    uri =
+      if String.ends_with?(uri.path, "/publickey") do
+        Map.put(uri, :path, String.replace(uri.path, "/publickey", ""))
+      else
+        uri
+      end
+
+    URI.to_string(uri)
   end
 
   def fetch_public_key(conn) do

lib/pleroma/upload.ex

@@ -228,7 +228,14 @@ defp url_from_spec(%__MODULE__{name: name}, base_url, {:file, path}) do
           ""
         end
 
-    [base_url, "media", path]
+    prefix =
+      if is_nil(Pleroma.Config.get([__MODULE__, :base_url])) do
+        "media"
+      else
+        ""
+      end
+
+    [base_url, prefix, path]
     |> Path.join()
   end
 

lib/pleroma/user.ex

@@ -592,12 +592,23 @@ def get_followers_query(user, page) do
   @spec get_followers_query(User.t()) :: Ecto.Query.t()
   def get_followers_query(user), do: get_followers_query(user, nil)
 
+  @spec get_followers(User.t(), pos_integer()) :: {:ok, list(User.t())}
   def get_followers(user, page \\ nil) do
     q = get_followers_query(user, page)
 
     {:ok, Repo.all(q)}
   end
 
+  @spec get_external_followers(User.t(), pos_integer()) :: {:ok, list(User.t())}
+  def get_external_followers(user, page \\ nil) do
+    q =
+      user
+      |> get_followers_query(page)
+      |> User.Query.build(%{external: true})
+
+    {:ok, Repo.all(q)}
+  end
+
   def get_followers_ids(user, page \\ nil) do
     q = get_followers_query(user, page)
 
@@ -918,14 +929,26 @@ def muted_notifications?(nil, _), do: false
   def muted_notifications?(user, %{ap_id: ap_id}),
     do: Enum.member?(user.info.muted_notifications, ap_id)
 
-  def blocks?(%User{info: info} = _user, %{ap_id: ap_id}) do
+  def blocks?(%User{} = user, %User{} = target) do
-    blocks = info.blocks
+    blocks_ap_id?(user, target) || blocks_domain?(user, target)
-    domain_blocks = info.domain_blocks
-    %{host: host} = URI.parse(ap_id)
-
-    Enum.member?(blocks, ap_id) || Enum.any?(domain_blocks, &(&1 == host))
   end
 
+  def blocks?(nil, _), do: false
+
+  def blocks_ap_id?(%User{} = user, %User{} = target) do
+    Enum.member?(user.info.blocks, target.ap_id)
+  end
+
+  def blocks_ap_id?(_, _), do: false
+
+  def blocks_domain?(%User{} = user, %User{} = target) do
+    domain_blocks = Pleroma.Web.ActivityPub.MRF.subdomains_regex(user.info.domain_blocks)
+    %{host: host} = URI.parse(target.ap_id)
+    Pleroma.Web.ActivityPub.MRF.subdomain_match?(domain_blocks, host)
+  end
+
+  def blocks_domain?(_, _), do: false
+
   def subscribed_to?(user, %{ap_id: ap_id}) do
     with %User{} = target <- get_cached_by_ap_id(ap_id) do
       Enum.member?(target.info.subscribers, user.ap_id)

lib/pleroma/user_invite_token.ex

@@ -74,7 +74,7 @@ def find_by_token!(token), do: Repo.get_by!(UserInviteToken, token: token)
 
   @spec find_by_token(token()) :: {:ok, UserInviteToken.t()} | nil
   def find_by_token(token) do
-    with invite <- Repo.get_by(UserInviteToken, token: token) do
+    with %UserInviteToken{} = invite <- Repo.get_by(UserInviteToken, token: token) do
       {:ok, invite}
     end
   end

lib/pleroma/web/activity_pub/activity_pub.ex

@@ -631,15 +631,26 @@ def fetch_user_activities(user, reading_user, params \\ %{}) do
       |> Map.put("pinned_activity_ids", user.info.pinned_activities)
 
     recipients =
+      user_activities_recipients(%{
+        "godmode" => params["godmode"],
+        "reading_user" => reading_user
+      })
+
+    fetch_activities(recipients, params)
+    |> Enum.reverse()
+  end
+
+  defp user_activities_recipients(%{"godmode" => true}) do
+    []
+  end
+
+  defp user_activities_recipients(%{"reading_user" => reading_user}) do
     if reading_user do
       ["https://www.w3.org/ns/activitystreams#Public"] ++
         [reading_user.ap_id | reading_user.following]
     else
       ["https://www.w3.org/ns/activitystreams#Public"]
     end
-
-    fetch_activities(recipients, params)
-    |> Enum.reverse()
   end
 
   defp restrict_since(query, %{"since_id" => ""}), do: query

lib/pleroma/web/activity_pub/mrf.ex

@@ -25,4 +25,14 @@ def get_policies do
   defp get_policies(policy) when is_atom(policy), do: [policy]
   defp get_policies(policies) when is_list(policies), do: policies
   defp get_policies(_), do: []
+
+  @spec subdomains_regex([String.t()]) :: [Regex.t()]
+  def subdomains_regex(domains) when is_list(domains) do
+    for domain <- domains, do: ~r(^#{String.replace(domain, "*.", "(.*\\.)*")}$)
+  end
+
+  @spec subdomain_match?([Regex.t()], String.t()) :: boolean()
+  def subdomain_match?(domains, host) do
+    Enum.any?(domains, fn domain -> Regex.match?(domain, host) end)
+  end
 end

lib/pleroma/web/activity_pub/mrf/simple_policy.ex

@@ -4,22 +4,29 @@
 
 defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do
   alias Pleroma.User
+  alias Pleroma.Web.ActivityPub.MRF
   @moduledoc "Filter activities depending on their origin instance"
-  @behaviour Pleroma.Web.ActivityPub.MRF
+  @behaviour MRF
 
   defp check_accept(%{host: actor_host} = _actor_info, object) do
-    accepts = Pleroma.Config.get([:mrf_simple, :accept])
+    accepts =
+      Pleroma.Config.get([:mrf_simple, :accept])
+      |> MRF.subdomains_regex()
 
     cond do
       accepts == [] -> {:ok, object}
       actor_host == Pleroma.Config.get([Pleroma.Web.Endpoint, :url, :host]) -> {:ok, object}
-      Enum.member?(accepts, actor_host) -> {:ok, object}
+      MRF.subdomain_match?(accepts, actor_host) -> {:ok, object}
       true -> {:reject, nil}
     end
   end
 
   defp check_reject(%{host: actor_host} = _actor_info, object) do
-    if Enum.member?(Pleroma.Config.get([:mrf_simple, :reject]), actor_host) do
+    rejects =
+      Pleroma.Config.get([:mrf_simple, :reject])
+      |> MRF.subdomains_regex()
+
+    if MRF.subdomain_match?(rejects, actor_host) do
       {:reject, nil}
     else
       {:ok, object}
@@ -31,8 +38,12 @@ defp check_media_removal(
          %{"type" => "Create", "object" => %{"attachment" => child_attachment}} = object
        )
        when length(child_attachment) > 0 do
+    media_removal =
+      Pleroma.Config.get([:mrf_simple, :media_removal])
+      |> MRF.subdomains_regex()
+
     object =
-      if Enum.member?(Pleroma.Config.get([:mrf_simple, :media_removal]), actor_host) do
+      if MRF.subdomain_match?(media_removal, actor_host) do
         child_object = Map.delete(object["object"], "attachment")
         Map.put(object, "object", child_object)
       else
@@ -51,8 +62,12 @@ defp check_media_nsfw(
            "object" => child_object
          } = object
        ) do
+    media_nsfw =
+      Pleroma.Config.get([:mrf_simple, :media_nsfw])
+      |> MRF.subdomains_regex()
+
     object =
-      if Enum.member?(Pleroma.Config.get([:mrf_simple, :media_nsfw]), actor_host) do
+      if MRF.subdomain_match?(media_nsfw, actor_host) do
         tags = (child_object["tag"] || []) ++ ["nsfw"]
         child_object = Map.put(child_object, "tag", tags)
         child_object = Map.put(child_object, "sensitive", true)
@@ -67,12 +82,12 @@ defp check_media_nsfw(
   defp check_media_nsfw(_actor_info, object), do: {:ok, object}
 
   defp check_ftl_removal(%{host: actor_host} = _actor_info, object) do
+    timeline_removal =
+      Pleroma.Config.get([:mrf_simple, :federated_timeline_removal])
+      |> MRF.subdomains_regex()
+
     object =
-      with true <-
+      with true <- MRF.subdomain_match?(timeline_removal, actor_host),
-             Enum.member?(
-               Pleroma.Config.get([:mrf_simple, :federated_timeline_removal]),
-               actor_host
-             ),
            user <- User.get_cached_by_ap_id(object["actor"]),
            true <- "https://www.w3.org/ns/activitystreams#Public" in object["to"] do
         to =
@@ -94,7 +109,11 @@ defp check_ftl_removal(%{host: actor_host} = _actor_info, object) do
   end
 
   defp check_report_removal(%{host: actor_host} = _actor_info, %{"type" => "Flag"} = object) do
-    if actor_host in Pleroma.Config.get([:mrf_simple, :report_removal]) do
+    report_removal =
+      Pleroma.Config.get([:mrf_simple, :report_removal])
+      |> MRF.subdomains_regex()
+
+    if MRF.subdomain_match?(report_removal, actor_host) do
       {:reject, nil}
     else
       {:ok, object}
@@ -104,7 +123,11 @@ defp check_report_removal(%{host: actor_host} = _actor_info, %{"type" => "Flag"}
   defp check_report_removal(_actor_info, object), do: {:ok, object}
 
   defp check_avatar_removal(%{host: actor_host} = _actor_info, %{"icon" => _icon} = object) do
-    if actor_host in Pleroma.Config.get([:mrf_simple, :avatar_removal]) do
+    avatar_removal =
+      Pleroma.Config.get([:mrf_simple, :avatar_removal])
+      |> MRF.subdomains_regex()
+
+    if MRF.subdomain_match?(avatar_removal, actor_host) do
       {:ok, Map.delete(object, "icon")}
     else
       {:ok, object}
@@ -114,7 +137,11 @@ defp check_avatar_removal(%{host: actor_host} = _actor_info, %{"icon" => _icon}
   defp check_avatar_removal(_actor_info, object), do: {:ok, object}
 
   defp check_banner_removal(%{host: actor_host} = _actor_info, %{"image" => _image} = object) do
-    if actor_host in Pleroma.Config.get([:mrf_simple, :banner_removal]) do
+    banner_removal =
+      Pleroma.Config.get([:mrf_simple, :banner_removal])
+      |> MRF.subdomains_regex()
+
+    if MRF.subdomain_match?(banner_removal, actor_host) do
       {:ok, Map.delete(object, "image")}
     else
       {:ok, object}

lib/pleroma/web/activity_pub/publisher.ex

@@ -87,18 +87,23 @@ defp should_federate?(inbox, public) do
     if public do
       true
     else
-      inbox_info = URI.parse(inbox)
+      %{host: host} = URI.parse(inbox)
-      !Enum.member?(Config.get([:instance, :quarantined_instances], []), inbox_info.host)
+
+      quarantined_instances =
+        Config.get([:instance, :quarantined_instances], [])
+        |> Pleroma.Web.ActivityPub.MRF.subdomains_regex()
+
+      !Pleroma.Web.ActivityPub.MRF.subdomain_match?(quarantined_instances, host)
     end
   end
 
+  @spec recipients(User.t(), Activity.t()) :: list(User.t()) | []
   defp recipients(actor, activity) do
-    followers =
+    {:ok, followers} =
       if actor.follower_address in activity.recipients do
-        {:ok, followers} = User.get_followers(actor)
+        User.get_external_followers(actor)
-        Enum.filter(followers, &(!&1.local))
       else
-        []
+        {:ok, []}
       end
 
     Pleroma.Web.Salmon.remote_users(actor, activity) ++ followers
@@ -112,6 +117,45 @@ defp get_cc_ap_ids(ap_id, recipients) do
     |> Enum.map(& &1.ap_id)
   end
 
+  @as_public "https://www.w3.org/ns/activitystreams#Public"
+
+  defp maybe_use_sharedinbox(%User{info: %{source_data: data}}),
+    do: (is_map(data["endpoints"]) && Map.get(data["endpoints"], "sharedInbox")) || data["inbox"]
+
+  @doc """
+  Determine a user inbox to use based on heuristics.  These heuristics
+  are based on an approximation of the ``sharedInbox`` rules in the
+  [ActivityPub specification][ap-sharedinbox].
+
+  Please do not edit this function (or its children) without reading
+  the spec, as editing the code is likely to introduce some breakage
+  without some familiarity.
+
+     [ap-sharedinbox]: https://www.w3.org/TR/activitypub/#shared-inbox-delivery
+  """
+  def determine_inbox(
+        %Activity{data: activity_data},
+        %User{info: %{source_data: data}} = user
+      ) do
+    to = activity_data["to"] || []
+    cc = activity_data["cc"] || []
+    type = activity_data["type"]
+
+    cond do
+      type == "Delete" ->
+        maybe_use_sharedinbox(user)
+
+      @as_public in to || @as_public in cc ->
+        maybe_use_sharedinbox(user)
+
+      length(to) + length(cc) > 1 ->
+        maybe_use_sharedinbox(user)
+
+      true ->
+        data["inbox"]
+    end
+  end
+
   @doc """
   Publishes an activity with BCC to all relevant peers.
   """
@@ -166,8 +210,8 @@ def publish(%User{} = actor, %Activity{} = activity) do
 
     recipients(actor, activity)
     |> Enum.filter(fn user -> User.ap_enabled?(user) end)
-    |> Enum.map(fn %{info: %{source_data: data}} ->
+    |> Enum.map(fn %User{} = user ->
-      (is_map(data["endpoints"]) && Map.get(data["endpoints"], "sharedInbox")) || data["inbox"]
+      determine_inbox(activity, user)
     end)
     |> Enum.uniq()
     |> Enum.filter(fn inbox -> should_federate?(inbox, public) end)

lib/pleroma/web/activity_pub/visibility.ex

@@ -8,14 +8,14 @@ defmodule Pleroma.Web.ActivityPub.Visibility do
   alias Pleroma.Repo
   alias Pleroma.User
 
+  @public "https://www.w3.org/ns/activitystreams#Public"
+
+  @spec is_public?(Object.t() | Activity.t() | map()) :: boolean()
   def is_public?(%Object{data: %{"type" => "Tombstone"}}), do: false
   def is_public?(%Object{data: data}), do: is_public?(data)
   def is_public?(%Activity{data: data}), do: is_public?(data)
   def is_public?(%{"directMessage" => true}), do: false
-
+  def is_public?(data), do: @public in (data["to"] ++ (data["cc"] || []))
-  def is_public?(data) do
-    "https://www.w3.org/ns/activitystreams#Public" in (data["to"] ++ (data["cc"] || []))
-  end
 
   def is_private?(activity) do
     with false <- is_public?(activity),
@@ -69,15 +69,14 @@ def entire_thread_visible_for_user?(%Activity{} = activity, %User{} = user) do
   end
 
   def get_visibility(object) do
-    public = "https://www.w3.org/ns/activitystreams#Public"
     to = object.data["to"] || []
     cc = object.data["cc"] || []
 
     cond do
-      public in to ->
+      @public in to ->
         "public"
 
-      public in cc ->
+      @public in cc ->
         "unlisted"
 
       # this should use the sql for the object's activity

lib/pleroma/web/admin_api/admin_api_controller.ex

@@ -82,6 +82,25 @@ def user_show(conn, %{"nickname" => nickname}) do
     end
   end
 
+  def list_user_statuses(conn, %{"nickname" => nickname} = params) do
+    godmode = params["godmode"] == "true" || params["godmode"] == true
+
+    with %User{} = user <- User.get_cached_by_nickname_or_id(nickname) do
+      {_, page_size} = page_params(params)
+
+      activities =
+        ActivityPub.fetch_user_activities(user, nil, %{
+          "limit" => page_size,
+          "godmode" => godmode
+        })
+
+      conn
+      |> json(StatusView.render("index.json", %{activities: activities, as: :activity}))
+    else
+      _ -> {:error, :not_found}
+    end
+  end
+
   def user_toggle_activation(conn, %{"nickname" => nickname}) do
     user = User.get_cached_by_nickname(nickname)
 
@@ -272,11 +291,13 @@ def invites(conn, _params) do
 
   @doc "Revokes invite by token"
   def revoke_invite(conn, %{"token" => token}) do
-    invite = UserInviteToken.find_by_token!(token)
+    with {:ok, invite} <- UserInviteToken.find_by_token(token),
-    {:ok, updated_invite} = UserInviteToken.update_invite(invite, %{used: true})
+         {:ok, updated_invite} = UserInviteToken.update_invite(invite, %{used: true}) do
-
       conn
       |> json(AccountView.render("invite.json", %{invite: updated_invite}))
+    else
+      nil -> {:error, :not_found}
+    end
   end
 
   @doc "Get a password reset token (base64 string) for given nickname"

lib/pleroma/web/admin_api/config.ex

@@ -84,6 +84,7 @@ defp do_convert(entity) when is_map(entity) do
   end
 
   defp do_convert({:dispatch, [entity]}), do: %{"tuple" => [":dispatch", [inspect(entity)]]}
+  defp do_convert({:partial_chain, entity}), do: %{"tuple" => [":partial_chain", inspect(entity)]}
 
   defp do_convert(entity) when is_tuple(entity),
     do: %{"tuple" => do_convert(Tuple.to_list(entity))}
@@ -113,11 +114,15 @@ def transform(entity), do: :erlang.term_to_binary(entity)
   defp do_transform(%Regex{} = entity) when is_map(entity), do: entity
 
   defp do_transform(%{"tuple" => [":dispatch", [entity]]}) do
-    cleaned_string = String.replace(entity, ~r/[^\w|^{:,[|^,|^[|^\]^}|^\/|^\.|^"]^\s/, "")
+    {dispatch_settings, []} = do_eval(entity)
-    {dispatch_settings, []} = Code.eval_string(cleaned_string, [], requires: [], macros: [])
     {:dispatch, [dispatch_settings]}
   end
 
+  defp do_transform(%{"tuple" => [":partial_chain", entity]}) do
+    {partial_chain, []} = do_eval(entity)
+    {:partial_chain, partial_chain}
+  end
+
   defp do_transform(%{"tuple" => entity}) do
     Enum.reduce(entity, {}, fn val, acc -> Tuple.append(acc, do_transform(val)) end)
   end
@@ -149,4 +154,9 @@ defp do_transform_string(value) do
       do: String.to_existing_atom("Elixir." <> value),
       else: value
   end
+
+  defp do_eval(entity) do
+    cleaned_string = String.replace(entity, ~r/[^\w|^{:,[|^,|^[|^\]^}|^\/|^\.|^"]^\s/, "")
+    Code.eval_string(cleaned_string, [], requires: [], macros: [])
+  end
 end

lib/pleroma/web/common_api/utils.ex

@@ -439,6 +439,13 @@ def maybe_notify_mentioned_recipients(
 
   def maybe_notify_mentioned_recipients(recipients, _), do: recipients
 
+  # Do not notify subscribers if author is making a reply
+  def maybe_notify_subscribers(recipients, %Activity{
+        object: %Object{data: %{"inReplyTo" => _ap_id}}
+      }) do
+    recipients
+  end
+
   def maybe_notify_subscribers(
         recipients,
         %Activity{data: %{"actor" => actor, "type" => type}} = activity

lib/pleroma/web/mastodon_api/mastodon_api_controller.ex

@@ -883,7 +883,10 @@ def favourited_by(%{assigns: %{user: user}} = conn, %{"id" => id}) do
     with %Activity{data: %{"object" => object}} <- Activity.get_by_id(id),
          %Object{data: %{"likes" => likes}} <- Object.normalize(object) do
       q = from(u in User, where: u.ap_id in ^likes)
-      users = Repo.all(q)
+
+      users =
+        Repo.all(q)
+        |> Enum.filter(&(not User.blocks?(user, &1)))
 
       conn
       |> put_view(AccountView)
@@ -897,7 +900,10 @@ def reblogged_by(%{assigns: %{user: user}} = conn, %{"id" => id}) do
     with %Activity{data: %{"object" => object}} <- Activity.get_by_id(id),
          %Object{data: %{"announcements" => announces}} <- Object.normalize(object) do
       q = from(u in User, where: u.ap_id in ^announces)
-      users = Repo.all(q)
+
+      users =
+        Repo.all(q)
+        |> Enum.filter(&(not User.blocks?(user, &1)))
 
       conn
       |> put_view(AccountView)

lib/pleroma/web/mastodon_api/views/account_view.ex

@@ -50,13 +50,13 @@ def render("relationship.json", %{user: %User{} = user, target: %User{} = target
       id: to_string(target.id),
       following: User.following?(user, target),
       followed_by: User.following?(target, user),
-      blocking: User.blocks?(user, target),
+      blocking: User.blocks_ap_id?(user, target),
-      blocked_by: User.blocks?(target, user),
+      blocked_by: User.blocks_ap_id?(target, user),
       muting: User.mutes?(user, target),
       muting_notifications: User.muted_notifications?(user, target),
       subscribing: User.subscribed_to?(user, target),
       requested: requested,
-      domain_blocking: false,
+      domain_blocking: User.blocks_domain?(user, target),
       showing_reblogs: User.showing_reblogs?(user, target),
       endorsed: false
     }

lib/pleroma/web/mastodon_api/views/status_view.ex

@@ -222,7 +222,7 @@ def render("status.json", %{activity: %{data: %{"object" => _object}} = activity
       if user.local do
         Pleroma.Web.Router.Helpers.o_status_url(Pleroma.Web.Endpoint, :notice, activity)
       else
-        object.data["external_url"] || object.data["id"]
+        object.data["url"] || object.data["external_url"] || object.data["id"]
       end
 
     %{

lib/pleroma/web/rich_media/parser.ex

@@ -55,8 +55,8 @@ def ttl(data, url) do
         ttl_setters: [MyModule]
   """
   def set_ttl_based_on_image({:ok, data}, url) do
-    with {:ok, nil} <- Cachex.ttl(:rich_media_cache, url) do
+    with {:ok, nil} <- Cachex.ttl(:rich_media_cache, url),
-      ttl = get_ttl_from_image(data, url)
+         ttl when is_number(ttl) <- get_ttl_from_image(data, url) do
       Cachex.expire_at(:rich_media_cache, url, ttl * 1000)
       {:ok, data}
     else
@@ -82,6 +82,7 @@ defp parse_url(url) do
 
       html
       |> maybe_parse()
+      |> Map.put(:url, url)
       |> clean_parsed_data()
       |> check_parsed_data()
     rescue

lib/pleroma/web/router.ex

@@ -154,22 +154,12 @@ defmodule Pleroma.Web.Router do
     post("/users/follow", AdminAPIController, :user_follow)
     post("/users/unfollow", AdminAPIController, :user_unfollow)
 
-    # TODO: to be removed at version 1.0
-    delete("/user", AdminAPIController, :user_delete)
-    post("/user", AdminAPIController, :user_create)
-
     delete("/users", AdminAPIController, :user_delete)
     post("/users", AdminAPIController, :user_create)
     patch("/users/:nickname/toggle_activation", AdminAPIController, :user_toggle_activation)
     put("/users/tag", AdminAPIController, :tag_users)
     delete("/users/tag", AdminAPIController, :untag_users)
 
-    # TODO: to be removed at version 1.0
-    get("/permission_group/:nickname", AdminAPIController, :right_get)
-    get("/permission_group/:nickname/:permission_group", AdminAPIController, :right_get)
-    post("/permission_group/:nickname/:permission_group", AdminAPIController, :right_add)
-    delete("/permission_group/:nickname/:permission_group", AdminAPIController, :right_delete)
-
     get("/users/:nickname/permission_group", AdminAPIController, :right_get)
     get("/users/:nickname/permission_group/:permission_group", AdminAPIController, :right_get)
     post("/users/:nickname/permission_group/:permission_group", AdminAPIController, :right_add)
@@ -190,13 +180,11 @@ defmodule Pleroma.Web.Router do
     post("/users/revoke_invite", AdminAPIController, :revoke_invite)
     post("/users/email_invite", AdminAPIController, :email_invite)
 
-    # TODO: to be removed at version 1.0
-    get("/password_reset", AdminAPIController, :get_password_reset)
-
     get("/users/:nickname/password_reset", AdminAPIController, :get_password_reset)
 
     get("/users", AdminAPIController, :list_users)
     get("/users/:nickname", AdminAPIController, :user_show)
+    get("/users/:nickname/statuses", AdminAPIController, :list_user_statuses)
 
     get("/reports", AdminAPIController, :list_reports)
     get("/reports/:id", AdminAPIController, :report_show)
@@ -663,6 +651,12 @@ defmodule Pleroma.Web.Router do
     end
   end
 
+  scope "/", Pleroma.Web.ActivityPub do
+    pipe_through(:activitypub)
+    post("/inbox", ActivityPubController, :inbox)
+    post("/users/:nickname/inbox", ActivityPubController, :inbox)
+  end
+
   scope "/relay", Pleroma.Web.ActivityPub do
     pipe_through(:ap_service_actor)
 
@@ -677,12 +671,6 @@ defmodule Pleroma.Web.Router do
     post("/inbox", ActivityPubController, :inbox)
   end
 
-  scope "/", Pleroma.Web.ActivityPub do
-    pipe_through(:activitypub)
-    post("/inbox", ActivityPubController, :inbox)
-    post("/users/:nickname/inbox", ActivityPubController, :inbox)
-  end
-
   scope "/.well-known", Pleroma.Web do
     pipe_through(:well_known)
 

lib/pleroma/web/streamer.ex

@@ -13,6 +13,7 @@ defmodule Pleroma.Web.Streamer do
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.ActivityPub
   alias Pleroma.Web.ActivityPub.Visibility
+  alias Pleroma.Web.CommonAPI
   alias Pleroma.Web.MastodonAPI.NotificationView
 
   @keepalive_interval :timer.seconds(30)
@@ -118,10 +119,14 @@ def handle_cast(
     topics
     |> Map.get("#{topic}:#{item.user_id}", [])
     |> Enum.each(fn socket ->
+      with %User{} = user <- User.get_cached_by_ap_id(socket.assigns[:user].ap_id),
+           true <- should_send?(user, item),
+           false <- CommonAPI.thread_muted?(user, item.activity) do
         send(
           socket.transport_pid,
           {:text, represent_notification(socket.assigns[:user], item)}
         )
+      end
     end)
 
     {:noreply, topics}
@@ -225,19 +230,37 @@ defp represent_notification(%User{} = user, %Notification{} = notify) do
     |> Jason.encode!()
   end
 
+  defp should_send?(%User{} = user, %Activity{} = item) do
+    blocks = user.info.blocks || []
+    mutes = user.info.mutes || []
+    reblog_mutes = user.info.muted_reblogs || []
+    domain_blocks = Pleroma.Web.ActivityPub.MRF.subdomains_regex(user.info.domain_blocks)
+
+    with parent when not is_nil(parent) <- Object.normalize(item),
+         true <- Enum.all?([blocks, mutes, reblog_mutes], &(item.actor not in &1)),
+         true <- Enum.all?([blocks, mutes], &(parent.data["actor"] not in &1)),
+         %{host: item_host} <- URI.parse(item.actor),
+         %{host: parent_host} <- URI.parse(parent.data["actor"]),
+         false <- Pleroma.Web.ActivityPub.MRF.subdomain_match?(domain_blocks, item_host),
+         false <- Pleroma.Web.ActivityPub.MRF.subdomain_match?(domain_blocks, parent_host),
+         true <- thread_containment(item, user) do
+      true
+    else
+      _ -> false
+    end
+  end
+
+  defp should_send?(%User{} = user, %Notification{activity: activity}) do
+    should_send?(user, activity)
+  end
+
   def push_to_socket(topics, topic, %Activity{data: %{"type" => "Announce"}} = item) do
     Enum.each(topics[topic] || [], fn socket ->
       # Get the current user so we have up-to-date blocks etc.
       if socket.assigns[:user] do
         user = User.get_cached_by_ap_id(socket.assigns[:user].ap_id)
-        blocks = user.info.blocks || []
-        mutes = user.info.mutes || []
-        reblog_mutes = user.info.muted_reblogs || []
 
-        with parent when not is_nil(parent) <- Object.normalize(item),
+        if should_send?(user, item) do
-             true <- Enum.all?([blocks, mutes, reblog_mutes], &(item.actor not in &1)),
-             true <- Enum.all?([blocks, mutes], &(parent.data["actor"] not in &1)),
-             true <- thread_containment(item, user) do
           send(socket.transport_pid, {:text, represent_update(item, user)})
         end
       else

lib/pleroma/web/twitter_api/controllers/util_controller.ex

@@ -17,7 +17,6 @@ defmodule Pleroma.Web.TwitterAPI.UtilController do
   alias Pleroma.Web
   alias Pleroma.Web.ActivityPub.ActivityPub
   alias Pleroma.Web.CommonAPI
-  alias Pleroma.Web.OStatus
   alias Pleroma.Web.WebFinger
 
   def help_test(conn, _params) do
@@ -60,7 +59,7 @@ def remote_follow(%{assigns: %{user: user}} = conn, %{"acct" => acct}) do
       %Activity{id: activity_id} = Activity.get_create_by_object_ap_id(object.data["id"])
       redirect(conn, to: "/notice/#{activity_id}")
     else
-      {err, followee} = OStatus.find_or_make_user(acct)
+      {err, followee} = User.get_or_fetch(acct)
       avatar = User.avatar_url(followee)
       name = followee.nickname
       id = followee.id

lib/pleroma/web/web_finger/web_finger.ex

@@ -86,11 +86,17 @@ def represent_user(user, "XML") do
     |> XmlBuilder.to_doc()
   end
 
-  defp get_magic_key(magic_key) do
+  defp get_magic_key("data:application/magic-public-key," <> magic_key) do
-    "data:application/magic-public-key," <> magic_key = magic_key
     {:ok, magic_key}
-  rescue
+  end
-    MatchError -> {:error, "Missing magic key data."}
+
+  defp get_magic_key(nil) do
+    Logger.debug("Undefined magic key.")
+    {:ok, nil}
+  end
+
+  defp get_magic_key(_) do
+    {:error, "Missing magic key data."}
   end
 
   defp webfinger_from_xml(doc) do
@@ -187,6 +193,7 @@ def find_lrdd_template(domain) do
     end
   end
 
+  @spec finger(String.t()) :: {:ok, map()} | {:error, any()}
   def finger(account) do
     account = String.trim_leading(account, "@")
 
@@ -220,8 +227,6 @@ def finger(account) do
       else
         with {:ok, doc} <- Jason.decode(body) do
           webfinger_from_json(doc)
-        else
-          {:error, e} -> e
         end
       end
     else

lib/pleroma/web/web_finger/web_finger_controller.ex

@@ -7,6 +7,7 @@ defmodule Pleroma.Web.WebFinger.WebFingerController do
 
   alias Pleroma.Web.WebFinger
 
+  plug(Pleroma.Plugs.SetFormatPlug)
   plug(Pleroma.Web.FederatingPlug)
 
   def host_meta(conn, _params) do
@@ -17,9 +18,8 @@ def host_meta(conn, _params) do
     |> send_resp(200, xml)
   end
 
-  def webfinger(conn, %{"resource" => resource}) do
+  def webfinger(%{assigns: %{format: format}} = conn, %{"resource" => resource})
-    case get_format(conn) do
+      when format in ["xml", "xrd+xml"] do
-      n when n in ["xml", "xrd+xml"] ->
     with {:ok, response} <- WebFinger.webfinger(resource, "XML") do
       conn
       |> put_resp_content_type("application/xrd+xml")
@@ -27,20 +27,19 @@ def webfinger(conn, %{"resource" => resource}) do
     else
       _e -> send_resp(conn, 404, "Couldn't find user")
     end
+  end
 
-      n when n in ["json", "jrd+json"] ->
+  def webfinger(%{assigns: %{format: format}} = conn, %{"resource" => resource})
+      when format in ["json", "jrd+json"] do
     with {:ok, response} <- WebFinger.webfinger(resource, "JSON") do
       json(conn, response)
     else
-          _e -> send_resp(conn, 404, "Couldn't find user")
+      _e ->
-        end
+        conn
-
+        |> put_status(404)
-      _ ->
+        |> json("Couldn't find user")
-        send_resp(conn, 404, "Unsupported format")
     end
   end
 
-  def webfinger(conn, _params) do
+  def webfinger(conn, _params), do: send_resp(conn, 400, "Bad Request")
-    send_resp(conn, 400, "Bad Request")
-  end
 end

mix.exs

@@ -143,7 +143,7 @@ defp deps do
       {:telemetry, "~> 0.3"},
       {:prometheus_ex, "~> 3.0"},
       {:prometheus_plugs, "~> 1.1"},
-      {:prometheus_phoenix, "~> 1.2"},
+      {:prometheus_phoenix, "~> 1.3"},
       {:prometheus_ecto, "~> 1.4"},
       {:recon, github: "ferd/recon", tag: "2.4.0"},
       {:quack, "~> 0.1.1"},

mix.lock

@@ -56,22 +56,22 @@
   "nimble_parsec": {:hex, :nimble_parsec, "0.5.0", "90e2eca3d0266e5c53f8fbe0079694740b9c91b6747f2b7e3c5d21966bba8300", [:mix], [], "hexpm"},
   "parse_trans": {:hex, :parse_trans, "3.3.0", "09765507a3c7590a784615cfd421d101aec25098d50b89d7aa1d66646bc571c1", [:rebar3], [], "hexpm"},
   "pbkdf2_elixir": {:hex, :pbkdf2_elixir, "0.12.3", "6706a148809a29c306062862c803406e88f048277f6e85b68faf73291e820b84", [:mix], [], "hexpm"},
-  "phoenix": {:hex, :phoenix, "1.4.8", "c72dc3adeb49c70eb963a0ea24f7a064ec1588e651e84e1b7ad5ed8253c0b4a2", [:mix], [{:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:phoenix_pubsub, "~> 1.1", [hex: :phoenix_pubsub, repo: "hexpm", optional: false]}, {:plug, "~> 1.8.1 or ~> 1.9", [hex: :plug, repo: "hexpm", optional: false]}, {:plug_cowboy, "~> 1.0 or ~> 2.0", [hex: :plug_cowboy, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm"},
+  "phoenix": {:hex, :phoenix, "1.4.9", "746d098e10741c334d88143d3c94cab1756435f94387a63441792e66ec0ee974", [:mix], [{:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:phoenix_pubsub, "~> 1.1", [hex: :phoenix_pubsub, repo: "hexpm", optional: false]}, {:plug, "~> 1.8.1 or ~> 1.9", [hex: :plug, repo: "hexpm", optional: false]}, {:plug_cowboy, "~> 1.0 or ~> 2.0", [hex: :plug_cowboy, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm"},
   "phoenix_ecto": {:hex, :phoenix_ecto, "4.0.0", "c43117a136e7399ea04ecaac73f8f23ee0ffe3e07acfcb8062fe5f4c9f0f6531", [:mix], [{:ecto, "~> 3.0", [hex: :ecto, repo: "hexpm", optional: false]}, {:phoenix_html, "~> 2.9", [hex: :phoenix_html, repo: "hexpm", optional: true]}, {:plug, "~> 1.0", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
   "phoenix_html": {:hex, :phoenix_html, "2.13.1", "fa8f034b5328e2dfa0e4131b5569379003f34bc1fafdaa84985b0b9d2f12e68b", [:mix], [{:plug, "~> 1.5", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
   "phoenix_pubsub": {:hex, :phoenix_pubsub, "1.1.2", "496c303bdf1b2e98a9d26e89af5bba3ab487ba3a3735f74bf1f4064d2a845a3e", [:mix], [], "hexpm"},
   "pleroma_job_queue": {:hex, :pleroma_job_queue, "0.2.0", "879e660aa1cebe8dc6f0aaaa6aa48b4875e89cd961d4a585fd128e0773b31a18", [:mix], [], "hexpm"},
   "plug": {:hex, :plug, "1.8.2", "0bcce1daa420f189a6491f3940cc77ea7fb1919761175c9c3b59800d897440fc", [:mix], [{:mime, "~> 1.0", [hex: :mime, repo: "hexpm", optional: false]}, {:plug_crypto, "~> 1.0", [hex: :plug_crypto, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4", [hex: :telemetry, repo: "hexpm", optional: true]}], "hexpm"},
-  "plug_cowboy": {:hex, :plug_cowboy, "2.0.2", "6055f16868cc4882b24b6e1d63d2bada94fb4978413377a3b32ac16c18dffba2", [:mix], [{:cowboy, "~> 2.5", [hex: :cowboy, repo: "hexpm", optional: false]}, {:plug, "~> 1.7", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
+  "plug_cowboy": {:hex, :plug_cowboy, "2.1.0", "b75768153c3a8a9e8039d4b25bb9b14efbc58e9c4a6e6a270abff1cd30cbe320", [:mix], [{:cowboy, "~> 2.5", [hex: :cowboy, repo: "hexpm", optional: false]}, {:plug, "~> 1.7", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
   "plug_crypto": {:hex, :plug_crypto, "1.0.0", "18e49317d3fa343f24620ed22795ec29d4a5e602d52d1513ccea0b07d8ea7d4d", [:mix], [], "hexpm"},
   "plug_static_index_html": {:hex, :plug_static_index_html, "1.0.0", "840123d4d3975585133485ea86af73cb2600afd7f2a976f9f5fd8b3808e636a0", [:mix], [{:plug, "~> 1.0", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
   "poison": {:hex, :poison, "3.1.0", "d9eb636610e096f86f25d9a46f35a9facac35609a7591b3be3326e99a0484665", [:mix], [], "hexpm"},
   "poolboy": {:hex, :poolboy, "1.5.2", "392b007a1693a64540cead79830443abf5762f5d30cf50bc95cb2c1aaafa006b", [:rebar3], [], "hexpm"},
   "postgrex": {:hex, :postgrex, "0.14.3", "5754dee2fdf6e9e508cbf49ab138df964278700b764177e8f3871e658b345a1e", [:mix], [{:connection, "~> 1.0", [hex: :connection, repo: "hexpm", optional: false]}, {:db_connection, "~> 2.0", [hex: :db_connection, repo: "hexpm", optional: false]}, {:decimal, "~> 1.5", [hex: :decimal, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}], "hexpm"},
-  "prometheus": {:hex, :prometheus, "4.2.2", "a830e77b79dc6d28183f4db050a7cac926a6c58f1872f9ef94a35cd989aceef8", [:mix, :rebar3], [], "hexpm"},
+  "prometheus": {:hex, :prometheus, "4.4.1", "1e96073b3ed7788053768fea779cbc896ddc3bdd9ba60687f2ad50b252ac87d6", [:mix, :rebar3], [], "hexpm"},
   "prometheus_ecto": {:hex, :prometheus_ecto, "1.4.1", "6c768ea9654de871e5b32fab2eac348467b3021604ebebbcbd8bcbe806a65ed5", [:mix], [{:ecto, "~> 2.0 or ~> 3.0", [hex: :ecto, repo: "hexpm", optional: false]}, {:prometheus_ex, "~> 1.1 or ~> 2.0 or ~> 3.0", [hex: :prometheus_ex, repo: "hexpm", optional: false]}], "hexpm"},
   "prometheus_ex": {:hex, :prometheus_ex, "3.0.5", "fa58cfd983487fc5ead331e9a3e0aa622c67232b3ec71710ced122c4c453a02f", [:mix], [{:prometheus, "~> 4.0", [hex: :prometheus, repo: "hexpm", optional: false]}], "hexpm"},
-  "prometheus_phoenix": {:hex, :prometheus_phoenix, "1.2.1", "964a74dfbc055f781d3a75631e06ce3816a2913976d1df7830283aa3118a797a", [:mix], [{:phoenix, "~> 1.3", [hex: :phoenix, repo: "hexpm", optional: false]}, {:prometheus_ex, "~> 1.3 or ~> 2.0 or ~> 3.0", [hex: :prometheus_ex, repo: "hexpm", optional: false]}], "hexpm"},
+  "prometheus_phoenix": {:hex, :prometheus_phoenix, "1.3.0", "c4b527e0b3a9ef1af26bdcfbfad3998f37795b9185d475ca610fe4388fdd3bb5", [:mix], [{:phoenix, "~> 1.4", [hex: :phoenix, repo: "hexpm", optional: false]}, {:prometheus_ex, "~> 1.3 or ~> 2.0 or ~> 3.0", [hex: :prometheus_ex, repo: "hexpm", optional: false]}], "hexpm"},
   "prometheus_plugs": {:hex, :prometheus_plugs, "1.1.5", "25933d48f8af3a5941dd7b621c889749894d8a1082a6ff7c67cc99dec26377c5", [:mix], [{:accept, "~> 0.1", [hex: :accept, repo: "hexpm", optional: false]}, {:plug, "~> 1.0", [hex: :plug, repo: "hexpm", optional: false]}, {:prometheus_ex, "~> 1.1 or ~> 2.0 or ~> 3.0", [hex: :prometheus_ex, repo: "hexpm", optional: false]}, {:prometheus_process_collector, "~> 1.1", [hex: :prometheus_process_collector, repo: "hexpm", optional: true]}], "hexpm"},
   "prometheus_process_collector": {:hex, :prometheus_process_collector, "1.4.0", "6dbd39e3165b9ef1c94a7a820e9ffe08479f949dcdd431ed4aaea7b250eebfde", [:rebar3], [{:prometheus, "~> 4.0", [hex: :prometheus, repo: "hexpm", optional: false]}], "hexpm"},
   "quack": {:hex, :quack, "0.1.1", "cca7b4da1a233757fdb44b3334fce80c94785b3ad5a602053b7a002b5a8967bf", [:mix], [{:poison, ">= 1.0.0", [hex: :poison, repo: "hexpm", optional: false]}, {:tesla, "~> 1.2.0", [hex: :tesla, repo: "hexpm", optional: false]}], "hexpm"},

test/fixtures/tesla_mock/kpherox@mstdn.jp.xml

@@ -0,0 +1,10 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<XRD xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0">
+  <Subject>acct:kPherox@mstdn.jp</Subject>
+  <Alias>https://mstdn.jp/@kPherox</Alias>
+  <Alias>https://mstdn.jp/users/kPherox</Alias>
+  <Link rel="http://webfinger.net/rel/profile-page" type="text/html" href="https://mstdn.jp/@kPherox"/>
+  <Link rel="http://schemas.google.com/g/2010#updates-from" type="application/atom+xml" href="https://mstdn.jp/users/kPherox.atom"/>
+  <Link rel="self" type="application/activity+json" href="https://mstdn.jp/users/kPherox"/>
+  <Link rel="http://ostatus.org/schema/1.0/subscribe" template="https://mstdn.jp/authorize_interaction?acct={uri}"/>
+</XRD>

test/fixtures/tesla_mock/wedistribute-article.json

@@ -0,0 +1,18 @@
+{
+  "@context": [
+    "https://www.w3.org/ns/activitystreams"
+  ],
+  "type": "Article",
+  "name": "The end is near: Mastodon plans to drop OStatus support",
+  "content": "<!-- wp:paragraph {\"dropCap\":true} -->\n<p class=\"has-drop-cap\">The days of OStatus are numbered. The venerable protocol has served as a glue between many different types of servers since the early days of the Fediverse, connecting StatusNet (now GNU Social) to Friendica, Hubzilla, Mastodon, and Pleroma.</p>\n<!-- /wp:paragraph -->\n\n<!-- wp:paragraph -->\n<p>Now that many fediverse platforms support ActivityPub as a successor protocol, Mastodon appears to be drawing a line in the sand. In <a href=\"https://www.patreon.com/posts/mastodon-2-9-and-28121681\">a Patreon update</a>, Eugen Rochko writes:</p>\n<!-- /wp:paragraph -->\n\n<!-- wp:quote -->\n<blockquote class=\"wp-block-quote\"><p>...OStatus...has overstayed its welcome in the code...and now that most of the network uses ActivityPub, it's time for it to go. </p><cite>Eugen Rochko, Mastodon creator</cite></blockquote>\n<!-- /wp:quote -->\n\n<!-- wp:paragraph -->\n<p>The <a href=\"https://github.com/tootsuite/mastodon/pull/11205\">pull request</a> to remove Pubsubhubbub and Salmon, two of the main components of OStatus, has already been merged into Mastodon's master branch.</p>\n<!-- /wp:paragraph -->\n\n<!-- wp:paragraph -->\n<p>Some projects will be left in the dark as a side effect of this. GNU Social and PostActiv, for example, both only communicate using OStatus. While <a href=\"https://mastodon.social/@dansup/102076573310057902\">some discussion</a> exists regarding adopting ActivityPub for GNU Social, and <a href=\"https://notabug.org/diogo/gnu-social/src/activitypub/plugins/ActivityPub\">a plugin is in development</a>, it hasn't been formally adopted yet. We just hope that the <a href=\"https://status.fsf.org/main/public\">Free Software Foundation's instance</a> gets updated in time!</p>\n<!-- /wp:paragraph -->",
+  "summary": "One of the largest platforms in the federated social web is dropping the protocol that it started with.",
+  "attributedTo": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog",
+  "url": "https://wedistribute.org/2019/07/mastodon-drops-ostatus/",
+  "to": [
+    "https://www.w3.org/ns/activitystreams#Public",
+    "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/followers"
+  ],
+  "id": "https://wedistribute.org/wp-json/pterotype/v1/object/85810",
+  "likes": "https://wedistribute.org/wp-json/pterotype/v1/object/85810/likes",
+  "shares": "https://wedistribute.org/wp-json/pterotype/v1/object/85810/shares"
+}

test/fixtures/tesla_mock/wedistribute-user.json

@@ -0,0 +1,31 @@
+{
+  "@context": [
+    "https://www.w3.org/ns/activitystreams",
+    "https://w3id.org/security/v1",
+    {
+      "manuallyApprovesFollowers": "as:manuallyApprovesFollowers"
+    }
+  ],
+  "type": "Organization",
+  "id": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog",
+  "following": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/following",
+  "followers": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/followers",
+  "liked": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/liked",
+  "inbox": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/inbox",
+  "outbox": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/outbox",
+  "name": "We Distribute",
+  "preferredUsername": "blog",
+  "summary": "<p>Connecting many threads in the federated web. We Distribute is an independent publication dedicated to the fediverse, decentralization, P2P technologies, and Free Software!</p>",
+  "url": "https://wedistribute.org/",
+  "publicKey": {
+    "id": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog#publicKey",
+    "owner": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog",
+    "publicKeyPem": "-----BEGIN PUBLIC KEY-----\r\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA1bmUJ+y8PS8JFVi0KugN\r\nFl4pLvLog3V2lsV9ftmCXpveB/WJx66Tr1fQLsU3qYvQFc8UPGWD52zV4RENR1SN\r\nx0O6T2f97KUbRM+Ckow7Jyjtssgl+Mqq8UBZQ/+H8I/1Vpvt5E5hUykhFgwzx9qg\r\nzoIA3OK7alOpQbSoKXo0QcOh6yTRUnMSRMJAgUoZJzzXI/FmH/DtKr7ziQ1T2KWs\r\nVs8mWnTb/OlCxiheLuMlmJNMF+lPyVthvMIxF6Z5gV9d5QAmASSCI628e6uH2EUF\r\nDEEF5jo+Z5ffeNv28953lrnM+VB/wTjl3tYA+zCQeAmUPksX3E+YkXGxj+4rxBAY\r\n8wIDAQAB\r\n-----END PUBLIC KEY-----"
+  },
+  "manuallyApprovesFollowers": false,
+  "icon": {
+    "url": "https://wedistribute.org/wp-content/uploads/2019/02/b067de423757a538.png",
+    "type": "Image",
+    "mediaType": "image/png"
+  }
+}

test/notification_test.exs

@@ -42,6 +42,28 @@ test "it creates a notification for subscribed users" do
 
       assert notification.user_id == subscriber.id
     end
+
+    test "does not create a notification for subscribed users if status is a reply" do
+      user = insert(:user)
+      other_user = insert(:user)
+      subscriber = insert(:user)
+
+      User.subscribe(subscriber, other_user)
+
+      {:ok, activity} = CommonAPI.post(user, %{"status" => "test post"})
+
+      {:ok, _reply_activity} =
+        CommonAPI.post(other_user, %{
+          "status" => "test reply",
+          "in_reply_to_status_id" => activity.id
+        })
+
+      user_notifications = Notification.for_user(user)
+      assert length(user_notifications) == 1
+
+      subscriber_notifications = Notification.for_user(subscriber)
+      assert Enum.empty?(subscriber_notifications)
+    end
   end
 
   describe "create_notification" do

test/object/fetcher_test.exs

@@ -110,6 +110,13 @@ test "it can fetch peertube videos" do
       assert object
     end
 
+    test "it can fetch wedistribute articles" do
+      {:ok, object} =
+        Fetcher.fetch_object_from_id("https://wedistribute.org/wp-json/pterotype/v1/object/85810")
+
+      assert object
+    end
+
     test "all objects with fake directions are rejected by the object fetcher" do
       assert {:error, _} =
                Fetcher.fetch_and_contain_remote_object_from_id(

test/plugs/authentication_plug_test.exs

@@ -9,7 +9,6 @@ defmodule Pleroma.Plugs.AuthenticationPlugTest do
   alias Pleroma.User
 
   import ExUnit.CaptureLog
-  import Mock
 
   setup %{conn: conn} do
     user = %User{
@@ -67,14 +66,13 @@ test "check pbkdf2 hash" do
       refute AuthenticationPlug.checkpw("test-password1", hash)
     end
 
+    @tag :skip_on_mac
     test "check sha512-crypt hash" do
       hash =
         "$6$9psBWV8gxkGOZWBz$PmfCycChoxeJ3GgGzwvhlgacb9mUoZ.KUXNCssekER4SJ7bOK53uXrHNb2e4i8yPFgSKyzaW9CcmrDXWIEMtD1"
 
-      with_mock :crypt, crypt: fn _password, password_hash -> password_hash end do
       assert AuthenticationPlug.checkpw("password", hash)
     end
-    end
 
     test "it returns false when hash invalid" do
       hash =

test/plugs/legacy_authentication_plug_test.exs

@@ -5,19 +5,18 @@
 defmodule Pleroma.Plugs.LegacyAuthenticationPlugTest do
   use Pleroma.Web.ConnCase
 
+  import Pleroma.Factory
+
   alias Pleroma.Plugs.LegacyAuthenticationPlug
   alias Pleroma.User
 
-  import Mock
-
   setup do
-    # password is "password"
+    user =
-    user = %User{
+      insert(:user,
-      id: 1,
+        password: "password",
-      name: "dude",
         password_hash:
           "$6$9psBWV8gxkGOZWBz$PmfCycChoxeJ3GgGzwvhlgacb9mUoZ.KUXNCssekER4SJ7bOK53uXrHNb2e4i8yPFgSKyzaW9CcmrDXWIEMtD1"
-    }
+      )
 
     %{user: user}
   end
@@ -36,6 +35,7 @@ test "it does nothing if a user is assigned", %{conn: conn, user: user} do
     assert ret_conn == conn
   end
 
+  @tag :skip_on_mac
   test "it authenticates the auth_user if present and password is correct and resets the password",
        %{
          conn: conn,
@@ -46,22 +46,12 @@ test "it authenticates the auth_user if present and password is correct and rese
       |> assign(:auth_credentials, %{username: "dude", password: "password"})
       |> assign(:auth_user, user)
 
-    conn =
+    conn = LegacyAuthenticationPlug.call(conn, %{})
-      with_mocks([
+
-        {:crypt, [], [crypt: fn _password, password_hash -> password_hash end]},
+    assert conn.assigns.user.id == user.id
-        {User, [],
-         [
-           reset_password: fn user, %{password: password, password_confirmation: password} ->
-             {:ok, user}
-           end
-         ]}
-      ]) do
-        LegacyAuthenticationPlug.call(conn, %{})
-      end
-
-    assert conn.assigns.user == user
   end
 
+  @tag :skip_on_mac
   test "it does nothing if the password is wrong", %{
     conn: conn,
     user: user

test/plugs/set_format_plug_test.exs

@@ -0,0 +1,38 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Plugs.SetFormatPlugTest do
+  use ExUnit.Case, async: true
+  use Plug.Test
+
+  alias Pleroma.Plugs.SetFormatPlug
+
+  test "set format from params" do
+    conn =
+      :get
+      |> conn("/cofe?_format=json")
+      |> SetFormatPlug.call([])
+
+    assert %{format: "json"} == conn.assigns
+  end
+
+  test "set format from header" do
+    conn =
+      :get
+      |> conn("/cofe")
+      |> put_private(:phoenix_format, "xml")
+      |> SetFormatPlug.call([])
+
+    assert %{format: "xml"} == conn.assigns
+  end
+
+  test "doesn't set format" do
+    conn =
+      :get
+      |> conn("/cofe")
+      |> SetFormatPlug.call([])
+
+    refute conn.assigns[:format]
+  end
+end

test/signature_test.exs

@@ -48,16 +48,14 @@ test "it returns key" do
 
     test "it returns error when not found user" do
       assert capture_log(fn ->
-               assert Signature.fetch_public_key(make_fake_conn("test-ap_id")) ==
+               assert Signature.fetch_public_key(make_fake_conn("test-ap_id")) == {:error, :error}
-                        {:error, :error}
              end) =~ "[error] Could not decode user"
     end
 
     test "it returns error if public key is empty" do
       user = insert(:user, %{info: %{source_data: %{"publicKey" => %{}}}})
 
-      assert Signature.fetch_public_key(make_fake_conn(user.ap_id)) ==
+      assert Signature.fetch_public_key(make_fake_conn(user.ap_id)) == {:error, :error}
-               {:error, :error}
     end
   end
 
@@ -65,8 +63,7 @@ test "it returns error if public key is empty" do
     test "it returns key" do
       ap_id = "https://mastodon.social/users/lambadalambda"
 
-      assert Signature.refetch_public_key(make_fake_conn(ap_id)) ==
+      assert Signature.refetch_public_key(make_fake_conn(ap_id)) == {:ok, @rsa_public_key}
-               {:ok, @rsa_public_key}
     end
 
     test "it returns error when not found user" do
@@ -105,4 +102,16 @@ test "it returns error" do
              ) == {:error, []}
     end
   end
+
+  describe "key_id_to_actor_id/1" do
+    test "it properly deduces the actor id for misskey" do
+      assert Signature.key_id_to_actor_id("https://example.com/users/1234/publickey") ==
+               "https://example.com/users/1234"
+    end
+
+    test "it properly deduces the actor id for mastodon and pleroma" do
+      assert Signature.key_id_to_actor_id("https://example.com/users/1234#main-key") ==
+               "https://example.com/users/1234"
+    end
+  end
 end

test/support/factory.ex

@@ -118,9 +118,12 @@ def direct_note_activity_factory do
   def note_activity_factory(attrs \\ %{}) do
     user = attrs[:user] || insert(:user)
     note = attrs[:note] || insert(:note, user: user)
-    attrs = Map.drop(attrs, [:user, :note])
 
-    data = %{
+    data_attrs = attrs[:data_attrs] || %{}
+    attrs = Map.drop(attrs, [:user, :note, :data_attrs])
+
+    data =
+      %{
         "id" => Pleroma.Web.ActivityPub.Utils.generate_activity_id(),
         "type" => "Create",
         "actor" => note.data["actor"],
@@ -129,6 +132,7 @@ def note_activity_factory(attrs \\ %{}) do
         "published" => DateTime.utc_now() |> DateTime.to_iso8601(),
         "context" => note.data["context"]
       }
+      |> Map.merge(data_attrs)
 
     %Pleroma.Activity{
       data: data,

test/support/http_request_mock.ex

@@ -301,6 +301,22 @@ def get("https://baptiste.gelez.xyz/~/PlumeDevelopment/this-month-in-plume-june-
      }}
   end
 
+  def get("https://wedistribute.org/wp-json/pterotype/v1/object/85810", _, _, _) do
+    {:ok,
+     %Tesla.Env{
+       status: 200,
+       body: File.read!("test/fixtures/tesla_mock/wedistribute-article.json")
+     }}
+  end
+
+  def get("https://wedistribute.org/wp-json/pterotype/v1/actor/-blog", _, _, _) do
+    {:ok,
+     %Tesla.Env{
+       status: 200,
+       body: File.read!("test/fixtures/tesla_mock/wedistribute-user.json")
+     }}
+  end
+
   def get("http://mastodon.example.org/users/admin", _, _, Accept: "application/activity+json") do
     {:ok,
      %Tesla.Env{
@@ -614,6 +630,15 @@ def get(
      }}
   end
 
+  def get(
+        "https://social.heldscal.la/.well-known/webfinger?resource=invalid_content@social.heldscal.la",
+        _,
+        _,
+        Accept: "application/xrd+xml,application/jrd+json"
+      ) do
+    {:ok, %Tesla.Env{status: 200, body: ""}}
+  end
+
   def get("http://framatube.org/.well-known/host-meta", _, _, _) do
     {:ok,
      %Tesla.Env{
@@ -915,6 +940,14 @@ def get("https://info.pleroma.site/activity3.json", _, _, _) do
     {:ok, %Tesla.Env{status: 404, body: ""}}
   end
 
+  def get("https://mstdn.jp/.well-known/webfinger?resource=acct:kpherox@mstdn.jp", _, _, _) do
+    {:ok,
+     %Tesla.Env{
+       status: 200,
+       body: File.read!("test/fixtures/tesla_mock/kpherox@mstdn.jp.xml")
+     }}
+  end
+
   def get(url, query, body, headers) do
     {:error,
      "Not implemented the mock response for get #{inspect(url)}, #{query}, #{inspect(body)}, #{

test/tasks/user_test.exs

@@ -5,6 +5,9 @@
 defmodule Mix.Tasks.Pleroma.UserTest do
   alias Pleroma.Repo
   alias Pleroma.User
+  alias Pleroma.Web.OAuth.Authorization
+  alias Pleroma.Web.OAuth.Token
+
   use Pleroma.DataCase
 
   import Pleroma.Factory
@@ -327,6 +330,13 @@ test "invite is revoked" do
       assert_received {:mix_shell, :info, [message]}
       assert message =~ "Invite for token #{invite.token} was revoked."
     end
+
+    test "it prints an error message when invite is not exist" do
+      Mix.Tasks.Pleroma.User.run(["revoke_invite", "foo"])
+
+      assert_received {:mix_shell, :error, [message]}
+      assert message =~ "No invite found"
+    end
   end
 
   describe "running delete_activities" do
@@ -337,6 +347,13 @@ test "activities are deleted" do
       assert_received {:mix_shell, :info, [message]}
       assert message == "User #{nickname} statuses deleted."
     end
+
+    test "it prints an error message when user is not exist" do
+      Mix.Tasks.Pleroma.User.run(["delete_activities", "foo"])
+
+      assert_received {:mix_shell, :error, [message]}
+      assert message =~ "No local user"
+    end
   end
 
   describe "running toggle_confirmed" do
@@ -364,6 +381,13 @@ test "user is not confirmed" do
       refute user.info.confirmation_pending
       refute user.info.confirmation_token
     end
+
+    test "it prints an error message when user is not exist" do
+      Mix.Tasks.Pleroma.User.run(["toggle_confirmed", "foo"])
+
+      assert_received {:mix_shell, :error, [message]}
+      assert message =~ "No local user"
+    end
   end
 
   describe "search" do
@@ -386,4 +410,64 @@ test "it returns users matching" do
                User.Search.search("moon fediverse", for_user: user) |> Enum.map(& &1.id)
     end
   end
+
+  describe "signing out" do
+    test "it deletes all user's tokens and authorizations" do
+      user = insert(:user)
+      insert(:oauth_token, user: user)
+      insert(:oauth_authorization, user: user)
+
+      assert Repo.get_by(Token, user_id: user.id)
+      assert Repo.get_by(Authorization, user_id: user.id)
+
+      :ok = Mix.Tasks.Pleroma.User.run(["sign_out", user.nickname])
+
+      refute Repo.get_by(Token, user_id: user.id)
+      refute Repo.get_by(Authorization, user_id: user.id)
+    end
+
+    test "it prints an error message when user is not exist" do
+      Mix.Tasks.Pleroma.User.run(["sign_out", "foo"])
+
+      assert_received {:mix_shell, :error, [message]}
+      assert message =~ "No local user"
+    end
+  end
+
+  describe "tagging" do
+    test "it add tags to a user" do
+      user = insert(:user)
+
+      :ok = Mix.Tasks.Pleroma.User.run(["tag", user.nickname, "pleroma"])
+
+      user = User.get_cached_by_nickname(user.nickname)
+      assert "pleroma" in user.tags
+    end
+
+    test "it prints an error message when user is not exist" do
+      Mix.Tasks.Pleroma.User.run(["tag", "foo"])
+
+      assert_received {:mix_shell, :error, [message]}
+      assert message =~ "Could not change user tags"
+    end
+  end
+
+  describe "untagging" do
+    test "it deletes tags from a user" do
+      user = insert(:user, tags: ["pleroma"])
+      assert "pleroma" in user.tags
+
+      :ok = Mix.Tasks.Pleroma.User.run(["untag", user.nickname, "pleroma"])
+
+      user = User.get_cached_by_nickname(user.nickname)
+      assert Enum.empty?(user.tags)
+    end
+
+    test "it prints an error message when user is not exist" do
+      Mix.Tasks.Pleroma.User.run(["untag", "foo"])
+
+      assert_received {:mix_shell, :error, [message]}
+      assert message =~ "Could not change user tags"
+    end
+  end
 end

test/test_helper.exs

@@ -2,7 +2,8 @@
 # Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
 # SPDX-License-Identifier: AGPL-3.0-only
 
-ExUnit.start()
+os_exclude = if :os.type() == {:unix, :darwin}, do: [skip_on_mac: true], else: []
+ExUnit.start(exclude: os_exclude)
 Ecto.Adapters.SQL.Sandbox.mode(Pleroma.Repo, :manual)
 Mox.defmock(Pleroma.ReverseProxy.ClientMock, for: Pleroma.ReverseProxy.Client)
 {:ok, _} = Application.ensure_all_started(:ex_machina)

test/upload/filter/dedupe_test.exs

@@ -25,7 +25,7 @@ test "adds shasum" do
 
     assert {
              :ok,
-             %Pleroma.Upload{id: @shasum, path: "#{@shasum}.jpg"}
+             %Pleroma.Upload{id: @shasum, path: @shasum <> ".jpg"}
            } = Dedupe.filter(upload)
   end
 end

test/upload_test.exs

@@ -122,24 +122,6 @@ test "returns a media url" do
       assert String.starts_with?(url, Pleroma.Web.base_url() <> "/media/")
     end
 
-    test "returns a media url with configured base_url" do
-      base_url = "https://cache.pleroma.social"
-
-      File.cp!("test/fixtures/image.jpg", "test/fixtures/image_tmp.jpg")
-
-      file = %Plug.Upload{
-        content_type: "image/jpg",
-        path: Path.absname("test/fixtures/image_tmp.jpg"),
-        filename: "image.jpg"
-      }
-
-      {:ok, data} = Upload.store(file, base_url: base_url)
-
-      assert %{"url" => [%{"href" => url}]} = data
-
-      assert String.starts_with?(url, base_url <> "/media/")
-    end
-
     test "copies the file to the configured folder with deduping" do
       File.cp!("test/fixtures/image.jpg", "test/fixtures/image_tmp.jpg")
 
@@ -266,4 +248,32 @@ test "escapes reserved uri characters" do
                "%3A%3F%23%5B%5D%40%21%24%26%5C%27%28%29%2A%2B%2C%3B%3D.jpg"
     end
   end
+
+  describe "Setting a custom base_url for uploaded media" do
+    setup do
+      Pleroma.Config.put([Pleroma.Upload, :base_url], "https://cache.pleroma.social")
+
+      on_exit(fn ->
+        Pleroma.Config.put([Pleroma.Upload, :base_url], nil)
+      end)
+    end
+
+    test "returns a media url with configured base_url" do
+      base_url = Pleroma.Config.get([Pleroma.Upload, :base_url])
+
+      File.cp!("test/fixtures/image.jpg", "test/fixtures/image_tmp.jpg")
+
+      file = %Plug.Upload{
+        content_type: "image/jpg",
+        path: Path.absname("test/fixtures/image_tmp.jpg"),
+        filename: "image.jpg"
+      }
+
+      {:ok, data} = Upload.store(file, base_url: base_url)
+
+      assert %{"url" => [%{"href" => url}]} = data
+
+      refute String.starts_with?(url, base_url <> "/media/")
+    end
+  end
 end

test/user_test.exs

@@ -824,6 +824,48 @@ test "blocks domains" do
       assert User.blocks?(user, collateral_user)
     end
 
+    test "does not block domain with same end" do
+      user = insert(:user)
+
+      collateral_user =
+        insert(:user, %{ap_id: "https://another-awful-and-rude-instance.com/user/bully"})
+
+      {:ok, user} = User.block_domain(user, "awful-and-rude-instance.com")
+
+      refute User.blocks?(user, collateral_user)
+    end
+
+    test "does not block domain with same end if wildcard added" do
+      user = insert(:user)
+
+      collateral_user =
+        insert(:user, %{ap_id: "https://another-awful-and-rude-instance.com/user/bully"})
+
+      {:ok, user} = User.block_domain(user, "*.awful-and-rude-instance.com")
+
+      refute User.blocks?(user, collateral_user)
+    end
+
+    test "blocks domain with wildcard for subdomain" do
+      user = insert(:user)
+
+      user_from_subdomain =
+        insert(:user, %{ap_id: "https://subdomain.awful-and-rude-instance.com/user/bully"})
+
+      user_with_two_subdomains =
+        insert(:user, %{
+          ap_id: "https://subdomain.second_subdomain.awful-and-rude-instance.com/user/bully"
+        })
+
+      user_domain = insert(:user, %{ap_id: "https://awful-and-rude-instance.com/user/bully"})
+
+      {:ok, user} = User.block_domain(user, "*.awful-and-rude-instance.com")
+
+      assert User.blocks?(user, user_from_subdomain)
+      assert User.blocks?(user, user_with_two_subdomains)
+      assert User.blocks?(user, user_domain)
+    end
+
     test "unblocks domains" do
       user = insert(:user)
       collateral_user = insert(:user, %{ap_id: "https://awful-and-rude-instance.com/user/bully"})

test/web/activity_pub/activity_pub_test.exs

@@ -6,11 +6,9 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubTest do
   use Pleroma.DataCase
   alias Pleroma.Activity
   alias Pleroma.Builders.ActivityBuilder
-  alias Pleroma.Instances
   alias Pleroma.Object
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.ActivityPub
-  alias Pleroma.Web.ActivityPub.Publisher
   alias Pleroma.Web.ActivityPub.Utils
   alias Pleroma.Web.CommonAPI
 
@@ -1083,113 +1081,6 @@ test "it can create a Flag activity" do
            } = activity
   end
 
-  describe "publish_one/1" do
-    test_with_mock "calls `Instances.set_reachable` on successful federation if `unreachable_since` is not specified",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://200.site/users/nick1/inbox"
-
-      assert {:ok, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
-
-      assert called(Instances.set_reachable(inbox))
-    end
-
-    test_with_mock "calls `Instances.set_reachable` on successful federation if `unreachable_since` is set",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://200.site/users/nick1/inbox"
-
-      assert {:ok, _} =
-               Publisher.publish_one(%{
-                 inbox: inbox,
-                 json: "{}",
-                 actor: actor,
-                 id: 1,
-                 unreachable_since: NaiveDateTime.utc_now()
-               })
-
-      assert called(Instances.set_reachable(inbox))
-    end
-
-    test_with_mock "does NOT call `Instances.set_reachable` on successful federation if `unreachable_since` is nil",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://200.site/users/nick1/inbox"
-
-      assert {:ok, _} =
-               Publisher.publish_one(%{
-                 inbox: inbox,
-                 json: "{}",
-                 actor: actor,
-                 id: 1,
-                 unreachable_since: nil
-               })
-
-      refute called(Instances.set_reachable(inbox))
-    end
-
-    test_with_mock "calls `Instances.set_unreachable` on target inbox on non-2xx HTTP response code",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://404.site/users/nick1/inbox"
-
-      assert {:error, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
-
-      assert called(Instances.set_unreachable(inbox))
-    end
-
-    test_with_mock "it calls `Instances.set_unreachable` on target inbox on request error of any kind",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://connrefused.site/users/nick1/inbox"
-
-      assert {:error, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
-
-      assert called(Instances.set_unreachable(inbox))
-    end
-
-    test_with_mock "does NOT call `Instances.set_unreachable` if target is reachable",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://200.site/users/nick1/inbox"
-
-      assert {:ok, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
-
-      refute called(Instances.set_unreachable(inbox))
-    end
-
-    test_with_mock "does NOT call `Instances.set_unreachable` if target instance has non-nil `unreachable_since`",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://connrefused.site/users/nick1/inbox"
-
-      assert {:error, _} =
-               Publisher.publish_one(%{
-                 inbox: inbox,
-                 json: "{}",
-                 actor: actor,
-                 id: 1,
-                 unreachable_since: NaiveDateTime.utc_now()
-               })
-
-      refute called(Instances.set_unreachable(inbox))
-    end
-  end
-
   test "fetch_activities/2 returns activities addressed to a list " do
     user = insert(:user)
     member = insert(:user)

test/web/activity_pub/mrf/mrf_test.exs

@@ -0,0 +1,46 @@
+defmodule Pleroma.Web.ActivityPub.MRFTest do
+  use ExUnit.Case, async: true
+  alias Pleroma.Web.ActivityPub.MRF
+
+  test "subdomains_regex/1" do
+    assert MRF.subdomains_regex(["unsafe.tld", "*.unsafe.tld"]) == [
+             ~r/^unsafe.tld$/,
+             ~r/^(.*\.)*unsafe.tld$/
+           ]
+  end
+
+  describe "subdomain_match/2" do
+    test "common domains" do
+      regexes = MRF.subdomains_regex(["unsafe.tld", "unsafe2.tld"])
+
+      assert regexes == [~r/^unsafe.tld$/, ~r/^unsafe2.tld$/]
+
+      assert MRF.subdomain_match?(regexes, "unsafe.tld")
+      assert MRF.subdomain_match?(regexes, "unsafe2.tld")
+
+      refute MRF.subdomain_match?(regexes, "example.com")
+    end
+
+    test "wildcard domains with one subdomain" do
+      regexes = MRF.subdomains_regex(["*.unsafe.tld"])
+
+      assert regexes == [~r/^(.*\.)*unsafe.tld$/]
+
+      assert MRF.subdomain_match?(regexes, "unsafe.tld")
+      assert MRF.subdomain_match?(regexes, "sub.unsafe.tld")
+      refute MRF.subdomain_match?(regexes, "anotherunsafe.tld")
+      refute MRF.subdomain_match?(regexes, "unsafe.tldanother")
+    end
+
+    test "wildcard domains with two subdomains" do
+      regexes = MRF.subdomains_regex(["*.unsafe.tld"])
+
+      assert regexes == [~r/^(.*\.)*unsafe.tld$/]
+
+      assert MRF.subdomain_match?(regexes, "unsafe.tld")
+      assert MRF.subdomain_match?(regexes, "sub.sub.unsafe.tld")
+      refute MRF.subdomain_match?(regexes, "sub.anotherunsafe.tld")
+      refute MRF.subdomain_match?(regexes, "sub.unsafe.tldanother")
+    end
+  end
+end

test/web/activity_pub/mrf/simple_policy_test.exs

@@ -49,6 +49,19 @@ test "has a matching host" do
 
       assert SimplePolicy.filter(local_message) == {:ok, local_message}
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :media_removal], ["*.remote.instance"])
+      media_message = build_media_message()
+      local_message = build_local_message()
+
+      assert SimplePolicy.filter(media_message) ==
+               {:ok,
+                media_message
+                |> Map.put("object", Map.delete(media_message["object"], "attachment"))}
+
+      assert SimplePolicy.filter(local_message) == {:ok, local_message}
+    end
   end
 
   describe "when :media_nsfw" do
@@ -74,6 +87,20 @@ test "has a matching host" do
 
       assert SimplePolicy.filter(local_message) == {:ok, local_message}
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :media_nsfw], ["*.remote.instance"])
+      media_message = build_media_message()
+      local_message = build_local_message()
+
+      assert SimplePolicy.filter(media_message) ==
+               {:ok,
+                media_message
+                |> put_in(["object", "tag"], ["foo", "nsfw"])
+                |> put_in(["object", "sensitive"], true)}
+
+      assert SimplePolicy.filter(local_message) == {:ok, local_message}
+    end
   end
 
   defp build_media_message do
@@ -106,6 +133,15 @@ test "has a matching host" do
       assert SimplePolicy.filter(report_message) == {:reject, nil}
       assert SimplePolicy.filter(local_message) == {:ok, local_message}
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :report_removal], ["*.remote.instance"])
+      report_message = build_report_message()
+      local_message = build_local_message()
+
+      assert SimplePolicy.filter(report_message) == {:reject, nil}
+      assert SimplePolicy.filter(local_message) == {:ok, local_message}
+    end
   end
 
   defp build_report_message do
@@ -146,6 +182,27 @@ test "has a matching host" do
       assert SimplePolicy.filter(local_message) == {:ok, local_message}
     end
 
+    test "match with wildcard domain" do
+      {actor, ftl_message} = build_ftl_actor_and_message()
+
+      ftl_message_actor_host =
+        ftl_message
+        |> Map.fetch!("actor")
+        |> URI.parse()
+        |> Map.fetch!(:host)
+
+      Config.put([:mrf_simple, :federated_timeline_removal], ["*." <> ftl_message_actor_host])
+      local_message = build_local_message()
+
+      assert {:ok, ftl_message} = SimplePolicy.filter(ftl_message)
+      assert actor.follower_address in ftl_message["to"]
+      refute actor.follower_address in ftl_message["cc"]
+      refute "https://www.w3.org/ns/activitystreams#Public" in ftl_message["to"]
+      assert "https://www.w3.org/ns/activitystreams#Public" in ftl_message["cc"]
+
+      assert SimplePolicy.filter(local_message) == {:ok, local_message}
+    end
+
     test "has a matching host but only as:Public in to" do
       {_actor, ftl_message} = build_ftl_actor_and_message()
 
@@ -192,6 +249,14 @@ test "has a matching host" do
 
       assert SimplePolicy.filter(remote_message) == {:reject, nil}
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :reject], ["*.remote.instance"])
+
+      remote_message = build_remote_message()
+
+      assert SimplePolicy.filter(remote_message) == {:reject, nil}
+    end
   end
 
   describe "when :accept" do
@@ -224,6 +289,16 @@ test "has a matching host" do
       assert SimplePolicy.filter(local_message) == {:ok, local_message}
       assert SimplePolicy.filter(remote_message) == {:ok, remote_message}
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :accept], ["*.remote.instance"])
+
+      local_message = build_local_message()
+      remote_message = build_remote_message()
+
+      assert SimplePolicy.filter(local_message) == {:ok, local_message}
+      assert SimplePolicy.filter(remote_message) == {:ok, remote_message}
+    end
   end
 
   describe "when :avatar_removal" do
@@ -251,6 +326,15 @@ test "has a matching host" do
 
       refute filtered["icon"]
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :avatar_removal], ["*.remote.instance"])
+
+      remote_user = build_remote_user()
+      {:ok, filtered} = SimplePolicy.filter(remote_user)
+
+      refute filtered["icon"]
+    end
   end
 
   describe "when :banner_removal" do
@@ -278,6 +362,15 @@ test "has a matching host" do
 
       refute filtered["image"]
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :banner_removal], ["*.remote.instance"])
+
+      remote_user = build_remote_user()
+      {:ok, filtered} = SimplePolicy.filter(remote_user)
+
+      refute filtered["image"]
+    end
   end
 
   defp build_local_message do

test/web/activity_pub/publisher_test.exs

@@ -0,0 +1,266 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ActivityPub.PublisherTest do
+  use Pleroma.DataCase
+
+  import Pleroma.Factory
+  import Tesla.Mock
+  import Mock
+
+  alias Pleroma.Activity
+  alias Pleroma.Instances
+  alias Pleroma.Web.ActivityPub.Publisher
+
+  @as_public "https://www.w3.org/ns/activitystreams#Public"
+
+  setup do
+    mock(fn env -> apply(HttpRequestMock, :request, [env]) end)
+    :ok
+  end
+
+  describe "determine_inbox/2" do
+    test "it returns sharedInbox for messages involving as:Public in to" do
+      user =
+        insert(:user, %{
+          info: %{source_data: %{"endpoints" => %{"sharedInbox" => "http://example.com/inbox"}}}
+        })
+
+      activity = %Activity{
+        data: %{"to" => [@as_public], "cc" => [user.follower_address]}
+      }
+
+      assert Publisher.determine_inbox(activity, user) == "http://example.com/inbox"
+    end
+
+    test "it returns sharedInbox for messages involving as:Public in cc" do
+      user =
+        insert(:user, %{
+          info: %{source_data: %{"endpoints" => %{"sharedInbox" => "http://example.com/inbox"}}}
+        })
+
+      activity = %Activity{
+        data: %{"cc" => [@as_public], "to" => [user.follower_address]}
+      }
+
+      assert Publisher.determine_inbox(activity, user) == "http://example.com/inbox"
+    end
+
+    test "it returns sharedInbox for messages involving multiple recipients in to" do
+      user =
+        insert(:user, %{
+          info: %{source_data: %{"endpoints" => %{"sharedInbox" => "http://example.com/inbox"}}}
+        })
+
+      user_two = insert(:user)
+      user_three = insert(:user)
+
+      activity = %Activity{
+        data: %{"cc" => [], "to" => [user.ap_id, user_two.ap_id, user_three.ap_id]}
+      }
+
+      assert Publisher.determine_inbox(activity, user) == "http://example.com/inbox"
+    end
+
+    test "it returns sharedInbox for messages involving multiple recipients in cc" do
+      user =
+        insert(:user, %{
+          info: %{source_data: %{"endpoints" => %{"sharedInbox" => "http://example.com/inbox"}}}
+        })
+
+      user_two = insert(:user)
+      user_three = insert(:user)
+
+      activity = %Activity{
+        data: %{"to" => [], "cc" => [user.ap_id, user_two.ap_id, user_three.ap_id]}
+      }
+
+      assert Publisher.determine_inbox(activity, user) == "http://example.com/inbox"
+    end
+
+    test "it returns sharedInbox for messages involving multiple recipients in total" do
+      user =
+        insert(:user, %{
+          info: %{
+            source_data: %{
+              "inbox" => "http://example.com/personal-inbox",
+              "endpoints" => %{"sharedInbox" => "http://example.com/inbox"}
+            }
+          }
+        })
+
+      user_two = insert(:user)
+
+      activity = %Activity{
+        data: %{"to" => [user_two.ap_id], "cc" => [user.ap_id]}
+      }
+
+      assert Publisher.determine_inbox(activity, user) == "http://example.com/inbox"
+    end
+
+    test "it returns inbox for messages involving single recipients in total" do
+      user =
+        insert(:user, %{
+          info: %{
+            source_data: %{
+              "inbox" => "http://example.com/personal-inbox",
+              "endpoints" => %{"sharedInbox" => "http://example.com/inbox"}
+            }
+          }
+        })
+
+      activity = %Activity{
+        data: %{"to" => [user.ap_id], "cc" => []}
+      }
+
+      assert Publisher.determine_inbox(activity, user) == "http://example.com/personal-inbox"
+    end
+  end
+
+  describe "publish_one/1" do
+    test_with_mock "calls `Instances.set_reachable` on successful federation if `unreachable_since` is not specified",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://200.site/users/nick1/inbox"
+
+      assert {:ok, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
+
+      assert called(Instances.set_reachable(inbox))
+    end
+
+    test_with_mock "calls `Instances.set_reachable` on successful federation if `unreachable_since` is set",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://200.site/users/nick1/inbox"
+
+      assert {:ok, _} =
+               Publisher.publish_one(%{
+                 inbox: inbox,
+                 json: "{}",
+                 actor: actor,
+                 id: 1,
+                 unreachable_since: NaiveDateTime.utc_now()
+               })
+
+      assert called(Instances.set_reachable(inbox))
+    end
+
+    test_with_mock "does NOT call `Instances.set_reachable` on successful federation if `unreachable_since` is nil",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://200.site/users/nick1/inbox"
+
+      assert {:ok, _} =
+               Publisher.publish_one(%{
+                 inbox: inbox,
+                 json: "{}",
+                 actor: actor,
+                 id: 1,
+                 unreachable_since: nil
+               })
+
+      refute called(Instances.set_reachable(inbox))
+    end
+
+    test_with_mock "calls `Instances.set_unreachable` on target inbox on non-2xx HTTP response code",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://404.site/users/nick1/inbox"
+
+      assert {:error, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
+
+      assert called(Instances.set_unreachable(inbox))
+    end
+
+    test_with_mock "it calls `Instances.set_unreachable` on target inbox on request error of any kind",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://connrefused.site/users/nick1/inbox"
+
+      assert {:error, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
+
+      assert called(Instances.set_unreachable(inbox))
+    end
+
+    test_with_mock "does NOT call `Instances.set_unreachable` if target is reachable",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://200.site/users/nick1/inbox"
+
+      assert {:ok, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
+
+      refute called(Instances.set_unreachable(inbox))
+    end
+
+    test_with_mock "does NOT call `Instances.set_unreachable` if target instance has non-nil `unreachable_since`",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://connrefused.site/users/nick1/inbox"
+
+      assert {:error, _} =
+               Publisher.publish_one(%{
+                 inbox: inbox,
+                 json: "{}",
+                 actor: actor,
+                 id: 1,
+                 unreachable_since: NaiveDateTime.utc_now()
+               })
+
+      refute called(Instances.set_unreachable(inbox))
+    end
+  end
+
+  describe "publish/2" do
+    test_with_mock "publishes an activity with BCC to all relevant peers.",
+                   Pleroma.Web.Federator.Publisher,
+                   [:passthrough],
+                   [] do
+      follower =
+        insert(:user,
+          local: false,
+          info: %{
+            ap_enabled: true,
+            source_data: %{"inbox" => "https://domain.com/users/nick1/inbox"}
+          }
+        )
+
+      actor = insert(:user, follower_address: follower.ap_id)
+      user = insert(:user)
+
+      {:ok, _follower_one} = Pleroma.User.follow(follower, actor)
+      actor = refresh_record(actor)
+
+      note_activity =
+        insert(:note_activity,
+          recipients: [follower.ap_id],
+          data_attrs: %{"bcc" => [user.ap_id]}
+        )
+
+      res = Publisher.publish(actor, note_activity)
+      assert res == :ok
+
+      assert called(
+               Pleroma.Web.Federator.Publisher.enqueue_one(Publisher, %{
+                 inbox: "https://domain.com/users/nick1/inbox",
+                 actor: actor,
+                 id: note_activity.data["id"]
+               })
+             )
+    end
+  end
+end

test/web/admin_api/admin_api_controller_test.exs

@@ -1010,6 +1010,17 @@ test "with token" do
                "uses" => 0
              }
     end
+
+    test "with invalid token" do
+      admin = insert(:user, info: %{is_admin: true})
+
+      conn =
+        build_conn()
+        |> assign(:user, admin)
+        |> post("/api/pleroma/admin/users/revoke_invite", %{"token" => "foo"})
+
+      assert json_response(conn, :not_found) == "Not found"
+    end
   end
 
   describe "GET /api/pleroma/admin/reports/:id" do
@@ -1560,7 +1571,8 @@ test "common config example", %{conn: conn} do
                 %{"tuple" => [":method", "Pleroma.Captcha.Kocaptcha"]},
                 %{"tuple" => [":seconds_valid", 60]},
                 %{"tuple" => [":path", ""]},
-                %{"tuple" => [":key1", nil]}
+                %{"tuple" => [":key1", nil]},
+                %{"tuple" => [":partial_chain", "&:hackney_connect.partial_chain/1"]}
               ]
             }
           ]
@@ -1576,7 +1588,8 @@ test "common config example", %{conn: conn} do
                      %{"tuple" => [":method", "Pleroma.Captcha.Kocaptcha"]},
                      %{"tuple" => [":seconds_valid", 60]},
                      %{"tuple" => [":path", ""]},
-                     %{"tuple" => [":key1", nil]}
+                     %{"tuple" => [":key1", nil]},
+                     %{"tuple" => [":partial_chain", "&:hackney_connect.partial_chain/1"]}
                    ]
                  }
                ]
@@ -1902,6 +1915,63 @@ test "queues key as atom", %{conn: conn} do
              }
     end
   end
+
+  describe "GET /api/pleroma/admin/users/:nickname/statuses" do
+    setup do
+      admin = insert(:user, info: %{is_admin: true})
+      user = insert(:user)
+
+      date1 = (DateTime.to_unix(DateTime.utc_now()) + 2000) |> DateTime.from_unix!()
+      date2 = (DateTime.to_unix(DateTime.utc_now()) + 1000) |> DateTime.from_unix!()
+      date3 = (DateTime.to_unix(DateTime.utc_now()) + 3000) |> DateTime.from_unix!()
+
+      insert(:note_activity, user: user, published: date1)
+      insert(:note_activity, user: user, published: date2)
+      insert(:note_activity, user: user, published: date3)
+
+      conn =
+        build_conn()
+        |> assign(:user, admin)
+
+      {:ok, conn: conn, user: user}
+    end
+
+    test "renders user's statuses", %{conn: conn, user: user} do
+      conn = get(conn, "/api/pleroma/admin/users/#{user.nickname}/statuses")
+
+      assert json_response(conn, 200) |> length() == 3
+    end
+
+    test "renders user's statuses with a limit", %{conn: conn, user: user} do
+      conn = get(conn, "/api/pleroma/admin/users/#{user.nickname}/statuses?page_size=2")
+
+      assert json_response(conn, 200) |> length() == 2
+    end
+
+    test "doesn't return private statuses by default", %{conn: conn, user: user} do
+      {:ok, _private_status} =
+        CommonAPI.post(user, %{"status" => "private", "visibility" => "private"})
+
+      {:ok, _public_status} =
+        CommonAPI.post(user, %{"status" => "public", "visibility" => "public"})
+
+      conn = get(conn, "/api/pleroma/admin/users/#{user.nickname}/statuses")
+
+      assert json_response(conn, 200) |> length() == 4
+    end
+
+    test "returns private statuses with godmode on", %{conn: conn, user: user} do
+      {:ok, _private_status} =
+        CommonAPI.post(user, %{"status" => "private", "visibility" => "private"})
+
+      {:ok, _public_status} =
+        CommonAPI.post(user, %{"status" => "public", "visibility" => "public"})
+
+      conn = get(conn, "/api/pleroma/admin/users/#{user.nickname}/statuses?godmode=true")
+
+      assert json_response(conn, 200) |> length() == 5
+    end
+  end
 end
 
 # Needed for testing

test/web/admin_api/config_test.exs

@@ -238,6 +238,14 @@ test "simple keyword" do
       assert Config.from_binary(binary) == [key: "value"]
     end
 
+    test "keyword with partial_chain key" do
+      binary =
+        Config.transform([%{"tuple" => [":partial_chain", "&:hackney_connect.partial_chain/1"]}])
+
+      assert binary == :erlang.term_to_binary(partial_chain: &:hackney_connect.partial_chain/1)
+      assert Config.from_binary(binary) == [partial_chain: &:hackney_connect.partial_chain/1]
+    end
+
     test "keyword" do
       binary =
         Config.transform([

test/web/federator_test.exs

@@ -22,6 +22,15 @@ defmodule Pleroma.Web.FederatorTest do
     :ok
   end
 
+  describe "Publisher.perform" do
+    test "call `perform` with unknown task" do
+      assert {
+               :error,
+               "Don't know what to do with this"
+             } = Pleroma.Web.Federator.Publisher.perform("test", :ok, :ok)
+    end
+  end
+
   describe "Publish an activity" do
     setup do
       user = insert(:user)

test/web/mastodon_api/account_view_test.exs

@@ -231,6 +231,16 @@ test "represent a relationship for the blocking and blocked user" do
                AccountView.render("relationship.json", %{user: user, target: other_user})
     end
 
+    test "represent a relationship for the user blocking a domain" do
+      user = insert(:user)
+      other_user = insert(:user, ap_id: "https://bad.site/users/other_user")
+
+      {:ok, user} = User.block_domain(user, "bad.site")
+
+      assert %{domain_blocking: true, blocking: false} =
+               AccountView.render("relationship.json", %{user: user, target: other_user})
+    end
+
     test "represent a relationship for the user with a pending follow request" do
       user = insert(:user)
       other_user = insert(:user, %{info: %User.Info{locked: true}})

test/web/mastodon_api/mastodon_api_controller_test.exs

@@ -2815,11 +2815,11 @@ test "returns rich-media card", %{conn: conn, user: user} do
 
       card_data = %{
         "image" => "http://ia.media-imdb.com/images/rock.jpg",
-        "provider_name" => "www.imdb.com",
+        "provider_name" => "example.com",
-        "provider_url" => "http://www.imdb.com",
+        "provider_url" => "https://example.com",
         "title" => "The Rock",
         "type" => "link",
-        "url" => "http://www.imdb.com/title/tt0117500/",
+        "url" => "https://example.com/ogp",
         "description" =>
           "Directed by Michael Bay. With Sean Connery, Nicolas Cage, Ed Harris, John Spencer.",
         "pleroma" => %{
@@ -2827,7 +2827,7 @@ test "returns rich-media card", %{conn: conn, user: user} do
             "image" => "http://ia.media-imdb.com/images/rock.jpg",
             "title" => "The Rock",
             "type" => "video.movie",
-            "url" => "http://www.imdb.com/title/tt0117500/",
+            "url" => "https://example.com/ogp",
             "description" =>
               "Directed by Michael Bay. With Sean Connery, Nicolas Cage, Ed Harris, John Spencer."
           }
@@ -2868,14 +2868,14 @@ test "replaces missing description with an empty string", %{conn: conn, user: us
                "title" => "Pleroma",
                "description" => "",
                "image" => nil,
-               "provider_name" => "pleroma.social",
+               "provider_name" => "example.com",
-               "provider_url" => "https://pleroma.social",
+               "provider_url" => "https://example.com",
-               "url" => "https://pleroma.social/",
+               "url" => "https://example.com/ogp-missing-data",
                "pleroma" => %{
                  "opengraph" => %{
                    "title" => "Pleroma",
                    "type" => "website",
-                   "url" => "https://pleroma.social/"
+                   "url" => "https://example.com/ogp-missing-data"
                  }
                }
              }
@@ -3768,6 +3768,38 @@ test "returns empty array when status has not been favorited yet", %{
 
       assert Enum.empty?(response)
     end
+
+    test "does not return users who have favorited the status but are blocked", %{
+      conn: %{assigns: %{user: user}} = conn,
+      activity: activity
+    } do
+      other_user = insert(:user)
+      {:ok, user} = User.block(user, other_user)
+
+      {:ok, _, _} = CommonAPI.favorite(activity.id, other_user)
+
+      response =
+        conn
+        |> assign(:user, user)
+        |> get("/api/v1/statuses/#{activity.id}/favourited_by")
+        |> json_response(:ok)
+
+      assert Enum.empty?(response)
+    end
+
+    test "does not fail on an unauthenticated request", %{conn: conn, activity: activity} do
+      other_user = insert(:user)
+      {:ok, _, _} = CommonAPI.favorite(activity.id, other_user)
+
+      response =
+        conn
+        |> assign(:user, nil)
+        |> get("/api/v1/statuses/#{activity.id}/favourited_by")
+        |> json_response(:ok)
+
+      [%{"id" => id}] = response
+      assert id == other_user.id
+    end
   end
 
   describe "GET /api/v1/statuses/:id/reblogged_by" do
@@ -3807,6 +3839,38 @@ test "returns empty array when status has not been reblogged yet", %{
 
       assert Enum.empty?(response)
     end
+
+    test "does not return users who have reblogged the status but are blocked", %{
+      conn: %{assigns: %{user: user}} = conn,
+      activity: activity
+    } do
+      other_user = insert(:user)
+      {:ok, user} = User.block(user, other_user)
+
+      {:ok, _, _} = CommonAPI.repeat(activity.id, other_user)
+
+      response =
+        conn
+        |> assign(:user, user)
+        |> get("/api/v1/statuses/#{activity.id}/reblogged_by")
+        |> json_response(:ok)
+
+      assert Enum.empty?(response)
+    end
+
+    test "does not fail on an unauthenticated request", %{conn: conn, activity: activity} do
+      other_user = insert(:user)
+      {:ok, _, _} = CommonAPI.repeat(activity.id, other_user)
+
+      response =
+        conn
+        |> assign(:user, nil)
+        |> get("/api/v1/statuses/#{activity.id}/reblogged_by")
+        |> json_response(:ok)
+
+      [%{"id" => id}] = response
+      assert id == other_user.id
+    end
   end
 
   describe "POST /auth/password, with valid parameters" do

test/web/mastodon_api/status_view_test.exs

@@ -300,6 +300,16 @@ test "attachments" do
     assert %{id: "2"} = StatusView.render("attachment.json", %{attachment: object})
   end
 
+  test "put the url advertised in the Activity in to the url attribute" do
+    id = "https://wedistribute.org/wp-json/pterotype/v1/object/85810"
+    [activity] = Activity.search(nil, id)
+
+    status = StatusView.render("status.json", %{activity: activity})
+
+    assert status.uri == id
+    assert status.url == "https://wedistribute.org/2019/07/mastodon-drops-ostatus/"
+  end
+
   test "a reblog" do
     user = insert(:user)
     activity = insert(:note_activity)

test/web/oauth/oauth_controller_test.exs

@@ -5,9 +5,7 @@
 defmodule Pleroma.Web.OAuth.OAuthControllerTest do
   use Pleroma.Web.ConnCase
   import Pleroma.Factory
-  import Mock
 
-  alias Pleroma.Registration
   alias Pleroma.Repo
   alias Pleroma.Web.OAuth.Authorization
   alias Pleroma.Web.OAuth.OAuthController
@@ -108,11 +106,10 @@ test "with user-bound registration, GET /oauth/<provider>/callback redirects to
         "state" => ""
       }
 
-      with_mock Pleroma.Web.Auth.Authenticator,
-        get_registration: fn _ -> {:ok, registration} end do
       conn =
-          get(
+        conn
-            conn,
+        |> assign(:ueberauth_auth, %{provider: registration.provider, uid: registration.uid})
+        |> get(
           "/oauth/twitter/callback",
           %{
             "oauth_token" => "G-5a3AAAAAAAwMH9AAABaektfSM",
@@ -125,11 +122,10 @@ test "with user-bound registration, GET /oauth/<provider>/callback redirects to
       assert response = html_response(conn, 302)
       assert redirected_to(conn) =~ ~r/#{redirect_uri}\?code=.+/
     end
-    end
 
     test "with user-unbound registration, GET /oauth/<provider>/callback renders registration_details page",
          %{app: app, conn: conn} do
-      registration = insert(:registration, user: nil)
+      user = insert(:user)
 
       state_params = %{
         "scope" => "read write",
@@ -138,11 +134,14 @@ test "with user-unbound registration, GET /oauth/<provider>/callback renders reg
         "state" => "a_state"
       }
 
-      with_mock Pleroma.Web.Auth.Authenticator,
-        get_registration: fn _ -> {:ok, registration} end do
       conn =
-          get(
+        conn
-            conn,
+        |> assign(:ueberauth_auth, %{
+          provider: "twitter",
+          uid: "171799000",
+          info: %{nickname: user.nickname, email: user.email, name: user.name, description: nil}
+        })
+        |> get(
           "/oauth/twitter/callback",
           %{
             "oauth_token" => "G-5a3AAAAAAAwMH9AAABaektfSM",
@@ -155,9 +154,8 @@ test "with user-unbound registration, GET /oauth/<provider>/callback renders reg
       assert response = html_response(conn, 200)
       assert response =~ ~r/name="op" type="submit" value="register"/
       assert response =~ ~r/name="op" type="submit" value="connect"/
-        assert response =~ Registration.email(registration)
+      assert response =~ user.email
-        assert response =~ Registration.nickname(registration)
+      assert response =~ user.nickname
-      end
     end
 
     test "on authentication error, GET /oauth/<provider>/callback redirects to `redirect_uri`", %{

test/web/rich_media/parser_test.exs

@@ -59,7 +59,8 @@ test "returns error when no metadata present" do
 
   test "doesn't just add a title" do
     assert Pleroma.Web.RichMedia.Parser.parse("http://example.com/non-ogp") ==
-             {:error, "Found metadata was invalid or incomplete: %{}"}
+             {:error,
+              "Found metadata was invalid or incomplete: %{url: \"http://example.com/non-ogp\"}"}
   end
 
   test "parses ogp" do
@@ -71,7 +72,7 @@ test "parses ogp" do
                 description:
                   "Directed by Michael Bay. With Sean Connery, Nicolas Cage, Ed Harris, John Spencer.",
                 type: "video.movie",
-                url: "http://www.imdb.com/title/tt0117500/"
+                url: "http://example.com/ogp"
               }}
   end
 
@@ -84,7 +85,7 @@ test "falls back to <title> when ogp:title is missing" do
                 description:
                   "Directed by Michael Bay. With Sean Connery, Nicolas Cage, Ed Harris, John Spencer.",
                 type: "video.movie",
-                url: "http://www.imdb.com/title/tt0117500/"
+                url: "http://example.com/ogp-missing-title"
               }}
   end
 
@@ -96,7 +97,8 @@ test "parses twitter card" do
                 site: "@flickr",
                 image: "https://farm6.staticflickr.com/5510/14338202952_93595258ff_z.jpg",
                 title: "Small Island Developing States Photo Submission",
-                description: "View the album on Flickr."
+                description: "View the album on Flickr.",
+                url: "http://example.com/twitter-card"
               }}
   end
 
@@ -120,7 +122,7 @@ test "parses OEmbed" do
                 thumbnail_width: 150,
                 title: "Bacon Lollys",
                 type: "photo",
-                url: "https://farm4.staticflickr.com/3040/2362225867_4a87ab8baf_b.jpg",
+                url: "http://example.com/oembed",
                 version: "1.0",
                 web_page: "https://www.flickr.com/photos/bees/2362225867/",
                 web_page_short_url: "https://flic.kr/p/4AK2sc",

test/web/streamer_test.exs

@@ -65,6 +65,63 @@ test "it sends notify to in the 'user:notification' stream", %{user: user, notif
       Streamer.stream("user:notification", notify)
       Task.await(task)
     end
+
+    test "it doesn't send notify to the 'user:notification' stream when a user is blocked", %{
+      user: user
+    } do
+      blocked = insert(:user)
+      {:ok, user} = User.block(user, blocked)
+
+      task = Task.async(fn -> refute_receive {:text, _}, 4_000 end)
+
+      Streamer.add_socket(
+        "user:notification",
+        %{transport_pid: task.pid, assigns: %{user: user}}
+      )
+
+      {:ok, activity} = CommonAPI.post(user, %{"status" => ":("})
+      {:ok, notif, _} = CommonAPI.favorite(activity.id, blocked)
+
+      Streamer.stream("user:notification", notif)
+      Task.await(task)
+    end
+
+    test "it doesn't send notify to the 'user:notification' stream when a thread is muted", %{
+      user: user
+    } do
+      user2 = insert(:user)
+      task = Task.async(fn -> refute_receive {:text, _}, 4_000 end)
+
+      Streamer.add_socket(
+        "user:notification",
+        %{transport_pid: task.pid, assigns: %{user: user}}
+      )
+
+      {:ok, activity} = CommonAPI.post(user, %{"status" => "super hot take"})
+      {:ok, activity} = CommonAPI.add_mute(user, activity)
+      {:ok, notif, _} = CommonAPI.favorite(activity.id, user2)
+      Streamer.stream("user:notification", notif)
+      Task.await(task)
+    end
+
+    test "it doesn't send notify to the 'user:notification' stream' when a domain is blocked", %{
+      user: user
+    } do
+      user2 = insert(:user, %{ap_id: "https://hecking-lewd-place.com/user/meanie"})
+      task = Task.async(fn -> refute_receive {:text, _}, 4_000 end)
+
+      Streamer.add_socket(
+        "user:notification",
+        %{transport_pid: task.pid, assigns: %{user: user}}
+      )
+
+      {:ok, user} = User.block_domain(user, "hecking-lewd-place.com")
+      {:ok, activity} = CommonAPI.post(user, %{"status" => "super hot take"})
+      {:ok, notif, _} = CommonAPI.favorite(activity.id, user2)
+
+      Streamer.stream("user:notification", notif)
+      Task.await(task)
+    end
   end
 
   test "it sends to public" do

test/web/web_finger/web_finger_controller_test.exs

@@ -19,6 +19,19 @@ defmodule Pleroma.Web.WebFinger.WebFingerControllerTest do
     :ok
   end
 
+  test "GET host-meta" do
+    response =
+      build_conn()
+      |> get("/.well-known/host-meta")
+
+    assert response.status == 200
+
+    assert response.resp_body ==
+             ~s(<?xml version="1.0" encoding="UTF-8"?><XRD xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0"><Link rel="lrdd" template="#{
+               Pleroma.Web.base_url()
+             }/.well-known/webfinger?resource={uri}" type="application/xrd+xml" /></XRD>)
+  end
+
   test "Webfinger JRD" do
     user = insert(:user)
 
@@ -30,6 +43,16 @@ test "Webfinger JRD" do
     assert json_response(response, 200)["subject"] == "acct:#{user.nickname}@localhost"
   end
 
+  test "it returns 404 when user isn't found (JSON)" do
+    result =
+      build_conn()
+      |> put_req_header("accept", "application/jrd+json")
+      |> get("/.well-known/webfinger?resource=acct:jimm@localhost")
+      |> json_response(404)
+
+    assert result == "Couldn't find user"
+  end
+
   test "Webfinger XML" do
     user = insert(:user)
 
@@ -41,6 +64,26 @@ test "Webfinger XML" do
     assert response(response, 200)
   end
 
+  test "it returns 404 when user isn't found (XML)" do
+    result =
+      build_conn()
+      |> put_req_header("accept", "application/xrd+xml")
+      |> get("/.well-known/webfinger?resource=acct:jimm@localhost")
+      |> response(404)
+
+    assert result == "Couldn't find user"
+  end
+
+  test "Sends a 404 when invalid format" do
+    user = insert(:user)
+
+    assert_raise Phoenix.NotAcceptableError, fn ->
+      build_conn()
+      |> put_req_header("accept", "text/html")
+      |> get("/.well-known/webfinger?resource=acct:#{user.nickname}@localhost")
+    end
+  end
+
   test "Sends a 400 when resource param is missing" do
     response =
       build_conn()

test/web/web_finger/web_finger_test.exs

@@ -40,6 +40,11 @@ test "works for ap_ids" do
   end
 
   describe "fingering" do
+    test "returns error when fails parse xml or json" do
+      user = "invalid_content@social.heldscal.la"
+      assert {:error, %Jason.DecodeError{}} = WebFinger.finger(user)
+    end
+
     test "returns the info for an OStatus user" do
       user = "shp@social.heldscal.la"
 
@@ -81,6 +86,20 @@ test "returns the correctly for json ostatus users" do
       assert data["subscribe_address"] == "https://gnusocial.de/main/ostatussub?profile={uri}"
     end
 
+    test "it work for AP-only user" do
+      user = "kpherox@mstdn.jp"
+
+      {:ok, data} = WebFinger.finger(user)
+
+      assert data["magic_key"] == nil
+      assert data["salmon"] == nil
+
+      assert data["topic"] == "https://mstdn.jp/users/kPherox.atom"
+      assert data["subject"] == "acct:kPherox@mstdn.jp"
+      assert data["ap_id"] == "https://mstdn.jp/users/kPherox"
+      assert data["subscribe_address"] == "https://mstdn.jp/authorize_interaction?acct={uri}"
+    end
+
     test "it works for friendica" do
       user = "lain@squeet.me"