Merge branch 'feature/upload-filter-exiftool' into 'develop'

Add Pleroma.Upload.Filter.Exiftool

Closes #1935

See merge request pleroma/pleroma!2744
This commit is contained in:
lain 2020-07-11 11:31:44 +00:00
commit 11dd29ef3f
8 changed files with 68 additions and 9 deletions

1
.gitignore vendored
View File

@ -5,6 +5,7 @@
/*.ez /*.ez
/test/uploads /test/uploads
/.elixir_ls /.elixir_ls
/test/fixtures/DSCN0010_tmp.jpg
/test/fixtures/test_tmp.txt /test/fixtures/test_tmp.txt
/test/fixtures/image_tmp.jpg /test/fixtures/image_tmp.jpg
/test/tmp/ /test/tmp/

View File

@ -58,6 +58,7 @@ unit-testing:
alias: postgres alias: postgres
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
script: script:
- apt-get update && apt-get install -y libimage-exiftool-perl
- mix deps.get - mix deps.get
- mix ecto.create - mix ecto.create
- mix ecto.migrate - mix ecto.migrate
@ -89,6 +90,7 @@ unit-testing-rum:
<<: *global_variables <<: *global_variables
RUM_ENABLED: "true" RUM_ENABLED: "true"
script: script:
- apt-get update && apt-get install -y libimage-exiftool-perl
- mix deps.get - mix deps.get
- mix ecto.create - mix ecto.create
- mix ecto.migrate - mix ecto.migrate

View File

@ -57,6 +57,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
- MRF (`EmojiStealPolicy`): New MRF Policy which allows to automatically download emojis from remote instances - MRF (`EmojiStealPolicy`): New MRF Policy which allows to automatically download emojis from remote instances
- Support pagination in emoji packs API (for packs and for files in pack) - Support pagination in emoji packs API (for packs and for files in pack)
- Support for viewing instances favicons next to posts and accounts - Support for viewing instances favicons next to posts and accounts
- Added Pleroma.Upload.Filter.Exiftool as an alternate EXIF stripping mechanism targeting GPS/location metadata.
<details> <details>
<summary>API Changes</summary> <summary>API Changes</summary>

View File

@ -33,7 +33,7 @@ ARG DATA=/var/lib/pleroma
RUN echo "http://nl.alpinelinux.org/alpine/latest-stable/community" >> /etc/apk/repositories &&\ RUN echo "http://nl.alpinelinux.org/alpine/latest-stable/community" >> /etc/apk/repositories &&\
apk update &&\ apk update &&\
apk add imagemagick ncurses postgresql-client &&\ apk add exiftool imagemagick ncurses postgresql-client &&\
adduser --system --shell /bin/false --home ${HOME} pleroma &&\ adduser --system --shell /bin/false --home ${HOME} pleroma &&\
mkdir -p ${DATA}/uploads &&\ mkdir -p ${DATA}/uploads &&\
mkdir -p ${DATA}/static &&\ mkdir -p ${DATA}/static &&\

View File

@ -548,14 +548,6 @@ config :ex_aws, :s3,
### Upload filters ### Upload filters
#### Pleroma.Upload.Filter.Mogrify
* `args`: List of actions for the `mogrify` command like `"strip"` or `["strip", "auto-orient", {"implode", "1"}]`.
#### Pleroma.Upload.Filter.Dedupe
No specific configuration.
#### Pleroma.Upload.Filter.AnonymizeFilename #### Pleroma.Upload.Filter.AnonymizeFilename
This filter replaces the filename (not the path) of an upload. For complete obfuscation, add This filter replaces the filename (not the path) of an upload. For complete obfuscation, add
@ -563,6 +555,20 @@ This filter replaces the filename (not the path) of an upload. For complete obfu
* `text`: Text to replace filenames in links. If empty, `{random}.extension` will be used. You can get the original filename extension by using `{extension}`, for example `custom-file-name.{extension}`. * `text`: Text to replace filenames in links. If empty, `{random}.extension` will be used. You can get the original filename extension by using `{extension}`, for example `custom-file-name.{extension}`.
#### Pleroma.Upload.Filter.Dedupe
No specific configuration.
#### Pleroma.Upload.Filter.Exiftool
This filter only strips the GPS and location metadata with Exiftool leaving color profiles and attributes intact.
No specific configuration.
#### Pleroma.Upload.Filter.Mogrify
* `args`: List of actions for the `mogrify` command like `"strip"` or `["strip", "auto-orient", {"implode", "1"}]`.
## Email ## Email
### Pleroma.Emails.Mailer ### Pleroma.Emails.Mailer

View File

@ -0,0 +1,18 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Upload.Filter.Exiftool do
@moduledoc """
Strips GPS related EXIF tags and overwrites the file in place.
Also strips or replaces filesystem metadata e.g., timestamps.
"""
@behaviour Pleroma.Upload.Filter
def filter(%Pleroma.Upload{tempfile: file, content_type: "image" <> _}) do
System.cmd("exiftool", ["-overwrite_original", "-gps:all=", file], parallelism: true)
:ok
end
def filter(_), do: :ok
end

BIN
test/fixtures/DSCN0010.jpg vendored Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 158 KiB

View File

@ -0,0 +1,31 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Upload.Filter.ExiftoolTest do
use Pleroma.DataCase
alias Pleroma.Upload.Filter
test "apply exiftool filter" do
File.cp!(
"test/fixtures/DSCN0010.jpg",
"test/fixtures/DSCN0010_tmp.jpg"
)
upload = %Pleroma.Upload{
name: "image_with_GPS_data.jpg",
content_type: "image/jpg",
path: Path.absname("test/fixtures/DSCN0010.jpg"),
tempfile: Path.absname("test/fixtures/DSCN0010_tmp.jpg")
}
assert Filter.Exiftool.filter(upload) == :ok
{exif_original, 0} = System.cmd("exiftool", ["test/fixtures/DSCN0010.jpg"])
{exif_filtered, 0} = System.cmd("exiftool", ["test/fixtures/DSCN0010_tmp.jpg"])
refute exif_original == exif_filtered
assert String.match?(exif_original, ~r/GPS/)
refute String.match?(exif_filtered, ~r/GPS/)
end
end