cphipps/watchstate
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Updated README.md and FAQ.md

Browse Source
This commit is contained in:
Abdulmhsen B. A. A
2022-06-13 10:13:11 +03:00
parent 179cc91a46
commit ed4577386f
2 changed files with 247 additions and 287 deletions
Download Patch File Download Diff File Expand all files Collapse all files

236
FAQ.md
View File

@@ -1,6 +1,6 @@
# FAQ
### Q: How to update play state for newly added media backend without overwriting my current play state?
### Q: How to sync play state to a new backend without overwriting current play state?
Add the backend and when asked, answer `no` for allow import. when you finish, then run the following command:
@@ -15,23 +15,19 @@ successful you can then enable the import feature if you want.
### Q: Is there support for Multi-user setup?
No, The tool is designed to work for single user. However, It's possible to run container for each user.
No, The tool is designed to work for single user. However, It's possible to run container for each user. You can use
single container for all users, however it's not really easy refer
to [issue #136](https://github.com/ArabCoders/watchstate/issues/136).
Note: for Plex managed users run the following command to extract each managed user access token.
#### Note
Note: for Plex home/managed users run the following command to extract each managed user access token.
```bash
$ docker exec -ti console backend:users:list --with-tokens -- [BACKEND_NAME]
```
For Jellyfin/Emby, you can use same api token and just replace the user id.
---
### Q: Sometimes newly added episodes or movies don't make it to webhook endpoint?
As stated in webhook limitation section sometimes media backends don't make it easy to receive those events, as such, to
complement webhooks, its good idea enable the scheduled tasks of import/export and let them run once in a while to
remap the data.
For Jellyfin/Emby, you can just generate new API tokens.
----
@@ -90,7 +86,7 @@ mods to fix their db entries.
----
### Q: I keep on seeing "Ignoring 'XXX'. No valid/supported external ids." in logs?
### Q: I keep on seeing "No valid/supported external ids." in logs?
This most likely means that the item is not matched in your media backend
@@ -98,24 +94,16 @@ This most likely means that the item is not matched in your media backend
* For plex click the (...), and click Fix match. after that refresh metadata.
For episodes, we support both external ids like movies and relative unique ids, To make episodes sync work at least one
of the following conditions has to be met:
of the following conditions has to be true:
* The episode should have external ids.
* The series should have external ids, to make relative unique id works.
* The parent show should have external ids, to make relative unique id works.
---
### Q: Does this tool require webhooks to work?
No, You can use the task scheduler or on demand sync if you want. However, we recommend the webhook method as it's the
most efficient method to update play state.
---
### Q: When i use jellyfin, i sometimes see double events?
This most likely a bug in the plugin [jf-webhook #113](https://github.com/jellyfin/jellyfin-plugin-webhook/issues/113),
Just reload the page make sure there is only one added watchstate endpoint.
No, You can use the task scheduler or on demand sync if you want.
---
@@ -147,7 +135,7 @@ entirely by running the following command
$ docker exec -ti watchstate console servers:edit --delete --key options.ignore -- [SERVER_NAME]
```
##### Notice
##### Note
While this feature works for manual/task scheduler for all supported backends, Jellyfin/Emby does not report library id
on webhook event. So, this feature will not work for them in webhook context and the items will be processed.
@@ -211,15 +199,15 @@ where `[QUERY_STRING]` is the keyword that you want to search for
---
### Q: How to the metadata about specific item?
### Q: How to get metadata about specific item id?
Use the following command:
```bash
$ docker exec -ti console server backend:search:id [BACKEND_NAME] [ITEM_ID]
$ docker exec -ti console server backend:search:id [BACKEND_NAME] [BACKEND_ITEM_ID]
```
where `[ITEM_ID]` refers to backend item id
where `[BACKEND_ITEM_ID]` refers to backend item id
### Optional flags
@@ -243,12 +231,12 @@ where `[LIBRARY_ID]` refers to backend library id
* `[-p, --percentage]` How much in percentage the title has to be in path to be marked as matched item. Default
to `50.0%`.
* `[-o, --output]` Set output mode, it can be `yaml`, `json` or `table`. Default to `table`.
* `[-m, --method]` Which algorithm to use, it can be `similarity`, or `levenshtein`. Default to `similarity`.
* `[-m, --method]` Which method to use, it can be `similarity`, or `levenshtein`. Default to `similarity`.
* `[--include-raw-response]` Will include backend response in main response body with `raw` key.
---
### Q: Is it possible to look for unmatched items?
### Q: How to look for unmatched items?
Use the `backend:library:unmatched` command. For example,
@@ -315,3 +303,191 @@ it's slower than `MemoryMapper`.
| Memory Usage | (✗) Higher | (✓) Lower |
| Matching Speed | (✓) Faster | (✗) Slower |
| DB Operations | (✓) Faster | (✗) Slower |
---
### Q: What environment variables supported?
| Key | Type | Description | Default |
|-----------------------|--------|---------------------------------------------------------------------------------|------------------------------------|
| WS_DATA_PATH | string | Where key data stored (config, db). | `${BASE_PATH}/var` |
| WS_TMP_DIR | string | Where temp data stored. (logs, cache). | `${WS_DATA_PATH}` |
| WS_TZ | string | Set timezone. | `UTC` |
| WS_CRON_IMPORT | bool | Enable import scheduled task. Value casted to bool. | `false` |
| WS_CRON_IMPORT_AT | string | When to run import scheduled task. Valid Cron Expression Expected. | `0 */1 * * *` (Every 1h) |
| WS_CRON_IMPORT_ARGS | string | Flags to pass to the import command. | `-v` |
| WS_CRON_EXPORT | bool | Enable export scheduled task. Value casted to bool. | `false` |
| WS_CRON_EXPORT_AT | string | When to run export scheduled task. Valid Cron Expression Expected. | `30 */1 * * *` (Every 1h 30m) |
| WS_CRON_EXPORT_ARGS | string | Flags to pass to the export command. | `-v` |
| WS_CRON_PUSH | bool | Enable push scheduled task. Value casted to bool. | `false` |
| WS_CRON_PUSH_AT | string | When to run push scheduled task. Valid Cron Expression Expected. | `*/10 * * * *` (Every 10m) |
| WS_CRON_PUSH_ARGS | string | Flags to pass to the push command. | `-v` |
| WS_LOGS_PRUNE_AFTER | string | Delete logs older than specified time. Set to `disable` to disable the pruning. | `-3 DAYS` |
| WS_LOGS_CONTEXT | bool | Add context to console output messages. | `false` |
| WS_LOGGER_FILE_ENABLE | bool | Save logs to file. | `true` |
| WS_LOGGER_FILE | string | Full path to log file. | `${WS_TMP_DIR}/logs/app.(Ymd).log` |
| WS_LOGGER_FILE_LEVEL | string | File Logger Level. | `ERROR` |
| WS_WEBHOOK_DEBUG | bool | If enabled, allow dumping request/webhook using `rdump` & `wdump` parameters. | `false` |
#### Container specific environment variables.
| Key | Type | Description | Default |
|------------------|---------|----------------------------------------------------------------------|---------|
| WS_DISABLE_CHOWN | integer | Do not change ownership for needed directories inside the container. | `0` |
| WS_DISABLE_HTTP | integer | Disable included HTTP Server. | `0` |
| WS_DISABLE_CRON | integer | Disable included Task Scheduler. | `0` |
| WS_DISABLE_CACHE | integer | Disable included Cache Server. | `0` |
| WS_UID | integer | Set container user id. | `1000` |
| WS_GID | integer | Set container group id. | `1000` |
---
### Q: How to add webhooks?
To add webhook for your server the URL will be dependent on how you exposed webhook frontend, but typically it will be
like this:
Directly to container: `http://localhost:8081/?apikey=[WEBHOOK_TOKEN]`
Via reverse proxy : `https://watchstate.domain.example/?apikey=[WEBHOOK_TOKEN]`.
If your media backend support sending headers then remove query parameter `?apikey=[WEBHOOK_TOKEN]`, and add this header
```http request
X-apikey: [WEBHOOK_TOKEN]
```
where `[WEBHOOK_TOKEN]` Should match the backend specific `webhook.token` value. to see the token for each server run
```bash
$ docker exec -ti watchstate console servers:view --servers-filter [SERVER_NAME] -- webhook.token
```
If you see 'Not configured, or invalid key.' or empty value. run the following command
```bash
$ docker exec -ti watchstate console servers:edit --regenerate-webhook-token -- [SERVER_NAME]
```
#### Emby (you need "Emby Premiere" to use webhooks).
Go to your Manage Emby Server > Server > Webhooks > (Click Add Webhook)
##### Webhook Url:
`http://localhost:8081/?apikey=[WEBHOOK_TOKEN]`
##### Webhook Events
Select the following events
* Playback events
* User events
Click `Add Webhook`
#### Plex (you need "PlexPass" to use webhooks)
Go to your plex Web UI > Settings > Your Account > Webhooks > (Click ADD WEBHOOK)
##### URL:
`http://localhost:8081/?apikey=[WEBHOOK_TOKEN]`
Click `Save Changes`
#### Note:
If you have multiple plex backends and use the same PlexPass account for all of them, you have to unify the API key, by
running the following command:
```bash
$ docker exec -ti watchstate console servers:unify plex
Plex global webhook API key is: [random_string]
```
The reason is due to the way plex handle webhooks, And to know which webhook request belong to which server we have to
identify the backends, The unify command will do the necessary adjustments to handle multi plex server setup. for more
information run.
```bash
$ docker exec -ti watchstate console help servers:unify
```
#### Jellyfin (Free)
go to your jellyfin dashboard > plugins > Catalog > install: Notifications > Webhook, restart your jellyfin. After that
go back again to dashboard > plugins > webhook. Add `Add Generic Destination`,
##### Webhook Name:
Choose whatever name you want. For example, `Watchstate-Webhook`
##### Webhook Url:
`http://localhost:8081`
##### Notification Type:
Select the following events
* Item Added
* User Data Saved
* Playback Start
* Playback Stop
##### User Filter:
* Select your user.
##### Item Type:
* Movies
* Episodes
### Send All Properties (ignores template)
Toggle this checkbox.
### Add Request Header
Key: `x-apikey`
Value: `[WEBHOOK_TOKEN]`
Click `save`
---
### Q: What are the webhook limitations?
Those are some Webhook limitations we discovered for the following media backends.
#### Plex
* Plex does not send webhooks events for "marked as Played/Unplayed".
* Sometimes does not send events if you add more than one item at time.
* If you have multi-user setup, Plex will still report the admin account user id as `1`.
* When you mark items as unwatched, Plex reset the date on the object.
#### Emby
* Emby does not send webhooks events for newly added
items. [See feature request](https://emby.media/community/index.php?/topic/97889-new-content-notification-webhook/)
* Emby webhook test event does not contain data. To test if your setup works, play something or do mark an item as
played or unplayed you should see changes reflected in `docker exec -ti watchstate console db:list`.
#### Jellyfin
* If you don't select a user id, the Plugin will send `itemAdd` event without user data, and will fail the check if
you happen to enable `webhook.match.user` for jellyfin.
* Sometimes jellyfin will fire webhook `itemAdd` event without the item being matched.
* Even if you select user id, sometimes `itemAdd` event will fire without user data.
---
### Q: Sometimes newly added episodes or movies don't make it to webhook endpoint?
As stated in webhook limitation section sometimes media backends don't make it easy to receive those events, as such, to
complement webhooks, you should enable import/export tasks by settings their respective environment variables in
your `docker-compose.yaml` file.

298
README.md
View File

@@ -1,10 +1,9 @@
# WatchState
WatchState is primarily commandline tool that can sync your play state cross your different media backends, without
relying on 3rd party services, this tool support `Jellyfin`, `Plex Media Server`and `Emby` out of the box.
It's also come with some goodies, like finding `mis-identified items` or `unmatched items`. `querying` your
backend for specific item.
WatchState is a tool that can sync your play state cross your media backends, without relying on 3rd party services,
this tool support `Jellyfin`, `Plex Media Server`and `Emby`. It's also come with some features, like
finding `mis-identified` or `unmatched` items, and the ability to
`search` your backend for specific `item id` or `title`.
# Install
@@ -17,15 +16,15 @@ services:
image: ghcr.io/arabcoders/watchstate:latest
container_name: watchstate
restart: unless-stopped
# For more environment variables please read at the bottom of this page.
# For information about supported environment variables head to FAQ.md page.
# works for both global and container specific environment variables.
environment:
- WS_UID=${UID:-1000} # Set container user id.
- WS_GID=${GID:-1000} # Set container group id.
ports:
- "8081:80" # webhook listener port
- "8081:80" # webhook listener port.
volumes:
- ${PWD}/:/config:rw # mount current directory to container /config directory.
- ${PWD}:/config:rw # mount current directory to container /config directory.
```
After creating your docker compose file, start the container.
@@ -34,289 +33,74 @@ After creating your docker compose file, start the container.
$ docker-compose pull && docker-compose up -d
```
# First time
# Adding backends
Run the following command to see all available commands you can also run help on each command to get more info.
after starting the container for the first time you need to add your backends, and to do so run the following command:
```bash
# Show all commands.
$ docker exec -ti watchstate console list
# Show help document for each command.
$ docker exec -ti watchstate console help state:import
$ docker exec -ti watchstate console servers:manage --add -- [BACKEND_NAME]
```
This command is interactive and will ask you for some questions to add your backend, if you want to edit the backend
config again or if you made mistake just run the same command without `--add` flag. After adding your backends, You
should import your current play state.
---
After starting the container, you have to add your media backends, to do so run the following command:
# Importing play state.
To import your current play state from backends that have import enabled, run the following command:
```bash
$ docker exec -ti watchstate console servers:manage --add -- [SERVER_NAME]
$ docker exec -ti watchstate console state:import -v
```
This command is interactive and will ask you for some questions to add your backend, you can run the command as many
times as you want, if you want to edit the config again or if you made mistake just run the same command without `--add`
flag. After adding your backends, You should import your current play state by running the following command.
This command will pull your play state from all your backends compare them. To selectively import specific backends use
the `[-s, --servers-filter]` flag which accept comma seperated list of backend names. For example,
```bash
$ docker exec -ti watchstate console state:import -vvf
$ docker exec -ti watchstate console state:import -v --servers-filter 'home_plex,home_jellyfin'
```
---
Now that you have imported your current play state enable the import task by adding the following environment variables
to your `docker-compose.yaml` file `WS_CRON_IMPORT=1`. By default, we have it disabled. for more environment variables
please refer to [Environment variables list](FAQ.md#q-what-environment-variables-supported).
# Pulling play state.
### Supported import methods
Now that you have imported your current play state, you can stop manually running the command, and rely on the tasks
scheduler and webhooks to keep update your play state. To start receiving webhook events from backends you need to do
few more steps.
* Scheduled Task.
* On demand sync.
* Webhooks.
### Enable webhooks events for specific backend.
### Note:
To see the backend specific webhook api key run the following command:
```bash
$ docker exec -ti watchstate console servers:view --servers-filter [SERVER_NAME] -- webhook.token
```
If you see 'Not configured, or invalid key.' or empty value. run the following command
```bash
$ docker exec -ti watchstate console servers:edit --regenerate-webhook-token -- [SERVER_NAME]
```
---
#### Notice:
If you have multiple plex backends and use the same PlexPass account for all of them, you have to unify the API key, by
running the following command:
```bash
$ docker exec -ti watchstate console servers:unify plex
Plex global webhook API key is: [random_string]
```
The reason is due to the way plex handle webhooks, And to know which webhook request belong to which server we have to
identify the backends, The unify command will do the necessary adjustments to handle multi plex server setup. for more
information run.
```bash
$ docker exec -ti watchstate console help servers:unify
```
---
If you don't want to/can't use webhooks and want to rely on task scheduler importing, then set the value
of `WS_CRON_IMPORT` to `1`. By default, we run the import command every hour. However, you can change the scheduled task
timer by adding another variable `WS_CRON_IMPORT_AT` and set its value to valid cron expression. for
example, `0 */2 * * *` it will run every two hours instead of 1 hour. If your backends and this tool are not on same
server it might consume a lot of bandwidth depending on how big is your library as it's pulls the entire server library
listing.
---
#### Notice
You should still have `WS_CRON_IMPORT` enabled to keep healthy relation between storage and backend changes.
Even if you use webhooks import method, you should still have import task scheduler enabled to keep healthy
relationship. and pick up any missed events.
---
# Exporting play state
To manually export your play state back to backends you can run the following command
To export your current play state to backends that have export enabled, run the following command
```bash
$ docker exec -ti watchstate console state:export -vv
$ docker exec -ti watchstate console state:export -v
```
to sync specific server/s, use the `[-s, --servers-filter]` which accept comma seperated list of server names.
This command will export your current play state. To selectively export to specific backends use
the `[-s, --servers-filter]` flag which accept comma seperated list of server names. For example,
```bash
$ docker exec -ti watchstate console state:export -vv --servers-filter 'server1,server2'
$ docker exec -ti watchstate console state:export -v --servers-filter 'home_plex,home_jellyfin'
```
To enable export scheduled task set the value of `WS_CRON_EXPORT` to `1`. By default, we run export every 90 minutes.
However, you can change the timer by adding another variable called `WS_CRON_EXPORT_AT` and set its value to valid cron
expression. for example, `0 */3 * * *` it will run every three hours instead of 90 minutes.
# Start receiving webhook events.
By default, the official container includes http server exposed at port `80`, we officially don't support HTTPS
inside the container for the HTTP server. However, for the adventurous people we expose port 443 as well, as such you
can customize the `docker/files/nginx.conf` to support SSL. and do the necessary adjustments.
#### Example nginx reverse proxy.
```nginx
server {
server_name watchstate.domain.example;
location / {
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Protocol $scheme;
proxy_set_header X-Forwarded-Host $http_host;
proxy_pass http://localhost:8081/;
}
}
```
### Adding webhook
To add webhook for your server the URL will be dependent on how you exposed webhook frontend, but typically it will be
like this:
#### Webhook URL
Via reverse proxy : `https://watchstate.domain.example/?apikey=[WEBHOOK_TOKEN]`.
Directly to container: `https://localhost:8081/?apikey=[WEBHOOK_TOKEN]`
If your media backend support sending headers then remove query parameter `?apikey=[WEBHOOK_TOKEN]`, and add this header
```http request
X-apikey: [WEBHOOK_TOKEN]
```
where `[WEBHOOK_TOKEN]` Should match the backend specific `webhook.token` value. Refer to the steps described
at **[Steps to enable webhook servers](#enable-webhooks-events-for-specific-backend)**.
# Configuring media backends to send webhook events.
#### Jellyfin (Free)
go to your jellyfin dashboard > plugins > Catalog > install: Notifications > Webhook, restart your jellyfin. After that
go back again to dashboard > plugins > webhook. Add `Add Generic Destination`,
##### Webhook Name:
Choose whatever name you want. For example, `Watchstate-Webhook`
##### Webhook Url:
`http://localhost:8081`
##### Notification Type:
Select the following events
* Item Added
* User Data Saved
* Playback Start
* Playback Stop
##### User Filter:
* Select your user.
##### Item Type:
* Movies
* Episodes
### Send All Properties (ignores template)
Toggle this checkbox.
### Add Request Header
Key: `x-apikey`
Value: `[WEBHOOK_TOKEN]`
Click `save`
#### Emby (you need "Emby Premiere" to use webhooks)
Go to your Manage Emby Server > Server > Webhooks > (Click Add Webhook)
##### Webhook Url:
`http://localhost:8081/?apikey=[WEBHOOK_TOKEN]`
##### Webhook Events
Select the following events
* Playback events
* User events
Click `Add Webhook`
#### Plex (you need "PlexPass" to use webhooks)
Go to your plex Web UI > Settings > Your Account > Webhooks > (Click ADD WEBHOOK)
##### URL:
`http://localhost:8081/?apikey=[WEBHOOK_TOKEN]`
Click `Save Changes`
Now that you have exported your current play state, enable the export task by adding the following environment variables
to your `docker-compose.yaml` file `WS_CRON_EXPORT=1`. By default, we have it disabled. for more environment variables
please refer to [Environment variables list](FAQ.md#q-what-environment-variables-supported).
---
# Known Webhook limitations
# Plex
* Plex does not send webhooks events for "marked as Played/Unplayed".
* Sometimes does not send events if you add more than one item at time.
* If you have multi-user setup, Plex will still report the admin account user id as `1`.
* When you mark items as unwatched, Plex reset the date on the object.
# Emby
* Emby does not send webhooks events for newly added
items. [See feature request](https://emby.media/community/index.php?/topic/97889-new-content-notification-webhook/)
* Emby webhook test event does not contain data. To test if your setup works, play something or do mark an item as
played or unplayed you should see changes reflected in `docker exec -ti watchstate console db:list`.
# Jellyfin
* If you don't select a user id, the Plugin will send `itemAdd` event without user data, and will fail the check if
you happen to enable `strict user match` for jellyfin.
* Sometimes jellyfin will fire webhook `itemAdd` event without the item being matched.
* Even if you select user id, sometimes `itemAdd` event will fire without user data.
----
# Environment variables.
| Key | Type | Description | Default |
|-----------------------|--------|---------------------------------------------------------------------------------|------------------------------------|
| WS_DATA_PATH | string | Where key data stored (config, db). | `${BASE_PATH}/var` |
| WS_TMP_DIR | string | Where temp data stored. (logs, cache). | `${WS_DATA_PATH}` |
| WS_TZ | string | Set timezone. | `UTC` |
| WS_CRON_IMPORT | bool | Enable import scheduled task. Value casted to bool. | `false` |
| WS_CRON_IMPORT_AT | string | When to run import scheduled task. Valid Cron Expression Expected. | `0 */1 * * *` (Every 1h) |
| WS_CRON_IMPORT_ARGS | string | Flags to pass to the import command. | `-v` |
| WS_CRON_EXPORT | bool | Enable export scheduled task. Value casted to bool. | `false` |
| WS_CRON_EXPORT_AT | string | When to run export scheduled task. Valid Cron Expression Expected. | `30 */1 * * *` (Every 1h 30m) |
| WS_CRON_EXPORT_ARGS | string | Flags to pass to the export command. | `-v` |
| WS_CRON_PUSH | bool | Enable push scheduled task. Value casted to bool. | `false` |
| WS_CRON_PUSH_AT | string | When to run push scheduled task. Valid Cron Expression Expected. | `*/10 * * * *` (Every 10m) |
| WS_CRON_PUSH_ARGS | string | Flags to pass to the push command. | `-v` |
| WS_LOGS_PRUNE_AFTER | string | Delete logs older than specified time. Set to `disable` to disable the pruning. | `-3 DAYS` |
| WS_LOGS_CONTEXT | bool | Add context to console output messages. | `false` |
| WS_LOGGER_FILE_ENABLE | bool | Save logs to file. | `true` |
| WS_LOGGER_FILE | string | Full path to log file. | `${WS_TMP_DIR}/logs/app.(Ymd).log` |
| WS_LOGGER_FILE_LEVEL | string | File Logger Level. | `ERROR` |
| WS_WEBHOOK_DEBUG | bool | If enabled, allow dumping request/webhook using `rdump` & `wdump` parameters. | `false` |
# Container specific environment variables.
| Key | Type | Description | Default |
|------------------|---------|----------------------------------------------------------------------|---------|
| WS_DISABLE_CHOWN | integer | Do not change ownership for needed directories inside the container. | `0` |
| WS_DISABLE_HTTP | integer | Disable included HTTP Server. | `0` |
| WS_DISABLE_CRON | integer | Disable included Task Scheduler. | `0` |
| WS_DISABLE_CACHE | integer | Disable included Cache Server. | `0` |
| WS_UID | integer | Set container user id. | `1000` |
| WS_GID | integer | Set container group id. | `1000` |
# FAQ
For some common questions, Take look at this [frequently asked questions](FAQ.md) page.
Take look at this [frequently asked questions](FAQ.md) page. to know more about this tool and how to enable webhook
support and answers to many questions.