cphipps/watchstate
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Migrated old webhook code into the new api server.

Browse Source
This commit is contained in:
Abdulmhsen B. A. A
2024-04-27 16:21:38 +03:00
parent 1677a58785
commit 8c4c5fc3e4
4 changed files with 89 additions and 302 deletions
Download Patch File Download Diff File Expand all files Collapse all files

126
FAQ.md
View File

@@ -340,26 +340,26 @@ $ docker exec -ti watchstate console system:tasks
### How to add webhooks?
To add webhook for your backend the URL will be dependent on how you exposed webhook frontend, but typically it will be
like this: `http://localhost:8080/?apikey=[WEBHOOK_TOKEN]`, or via reverse proxy `https://watchstate.domain.example/?apikey=[WEBHOOK_TOKEN]`.
The Webhook URL is backend specific, the request path is `/v1/api/backends/[BACKEND_NAME]/webhook?apikey=[APIKEY]`,
Where `[BACKEND_NAME]` is the name of the backend you want to add webhook for, and `[APIKEY]` is the global api key
which you can get via the `system:apikey` command. Typically, the full path is `http://localhost:8080/v1/api/backends/[BACKEND_NAME]/webhook?apikey=[APIKEY]`. if the tool
port is directly exposed or via the reverse proxy you have setup.
If your media backend support sending headers then remove query parameter `?apikey=[WEBHOOK_TOKEN]`, and add this header
If your media backend support sending headers then remove query parameter `?apikey=[APIKEY]`, and add this header
```
x-apikey: [WEBHOOK_TOKEN]
Authorization: Bearer [APIKEY]
```
where `[WEBHOOK_TOKEN]` Should match the backend `webhook.token` value. To see your webhook token for each backend run:
To see your global api key run the following command:
```bash
$ docker exec -ti watchstate console config:view webhook.token
$ docker exec -ti watchstate console system:apikey
```
If you see 'Not configured, or invalid key.' or empty value. run the following command
```bash
$ docker exec -ti watchstate console config:edit --regenerate-webhook-token -s backend_name
```
> [!NOTE]
> You will keep seeing the `webhook.token` key, it's being kept for backward compatibility, and will be removed in the
> future.
-----
@@ -367,9 +367,16 @@ $ docker exec -ti watchstate console config:edit --regenerate-webhook-token -s b
Go to your Manage Emby Server > Server > Webhooks > (Click Add Webhook)
##### Webhook Url:
##### Webhook/Notifications URL:
`http://localhost:8080/?apikey=[WEBHOOK_TOKEN]`
`http://localhost:8080/v1/api/backends/[BACKEND_NAME]/webhook?apikey=[APIKEY]`
* Replace `[BACKEND_NAME]` with the name you have chosen for your backend.
* Replace `[APIKEY]` with the global apikey.
##### Request content type (Emby v4.9+):
`application/json`
##### Webhook Events:
@@ -393,11 +400,7 @@ Go to your Manage Emby Server > Server > Webhooks > (Click Add Webhook)
* Select libraries that you want to sync or leave it blank for all libraries.
Click `Add Webhook`
> [!NOTE]
> Emby version 4.9 replaced webhooks with Notification system, the system is somewhat similar to webhooks,
> There is an extra option called `Request content type` you should set it to `application/json`.
Click `Add Webhook / Save`
-----
@@ -407,31 +410,21 @@ Go to your Plex Web UI > Settings > Your Account > Webhooks > (Click ADD WEBHOOK
##### URL:
`http://localhost:8080/?apikey=[WEBHOOK_TOKEN]`
`http://localhost:8080/v1/api/backends/[BACKEND_NAME]/webhook?apikey=[APIKEY]`
* Replace `[BACKEND_NAME]` with the name you have chosen for your backend.
* Replace `[APIKEY]` with the global apikey.
> [!NOTE]
> If you use multiple plex servers and use the same PlexPass account for all of them, You have to add each backend
> using the same method above, while enabling `limit webhook events to` `selected user` and `backend unique id`.
> Essentially, this method replaced the old unified webhook.token for backends.
Click `Save Changes`
> [!IMPORTANT]
> If you use multiple plex servers and use the same PlexPass account for all of them, you have to unify the API key, by
> running the following command:
```bash
$ docker exec -ti watchstate console config:unify plex
Plex global webhook API key is: [random_string]
```
The reason is due to the way plex handle webhooks, And to know which webhook request belong to which backend we have to
identify the backends.
The unify command will do the necessary adjustments to handle multiple plex servers setup. for more information run.
```bash
$ docker exec -ti watchstate console help config:unify
```
> [!IMPORTANT]
> If you share your plex server with other users, i,e. `Home/managed users`, you have to enable match user id, otherwise
> their play state
> will end up changing your play state. Plex will still send their events. But with match user id they will be ignored.
> their play state will end up changing your play state.
-----
@@ -446,7 +439,9 @@ go back again to dashboard > plugins > webhook. Add `Add Generic Destination`,
##### Webhook Url:
`http://localhost:8080`
`http://localhost:8080/v1/api/backends/[BACKEND_NAME]/webhook`
* Replace `[BACKEND_NAME]` with the name you have chosen for your backend.
##### Notification Type:
@@ -470,10 +465,12 @@ Toggle this checkbox.
### Add Request Header
* Key: `x-apikey`
* Value: `[WEBHOOK_TOKEN]`
* Key: `Authorization`
* Value: `Bearer [APIKEY]`
Click `save`
Replace `[APIKEY]` with the global apikey.
Click `Save`
---
@@ -524,15 +521,15 @@ server need to send correct fastcgi environment variables. Example caddy file
https://watchstate.example.org {
# Change "172.23.1.2" to your watchstate container ip e.g. "172.23.20.20"
reverse_proxy 172.23.1.2:9000 {
transport fastcgi {
root /opt/app/public
env DOCUMENT_ROOT /opt/app/public
env SCRIPT_FILENAME /opt/app/public/index.php
env X_REQUEST_ID "{http.request.uuid}"
split .php
}
}
reverse_proxy 172.23.1.2:9000 {
transport fastcgi {
root /opt/app/public
env DOCUMENT_ROOT /opt/app/public
env SCRIPT_FILENAME /opt/app/public/index.php
env X_REQUEST_ID "{http.request.uuid}"
split .php
}
}
}
```
@@ -543,22 +540,12 @@ https://watchstate.example.org {
Set this environment variable in your `docker-compose.yaml` file `WS_DISABLE_CACHE` with value of `1`.
to use external redis server you need to alter the value of `WS_CACHE_URL` environment variable. the format for this
variable is `redis://host:port?password=auth&db=db_num`, for example to use redis from another container you could use
something like `redis://redis:6379?password=my_secert_password&db=8`. We only support `redis` at the moment.
something like `redis://172.23.1.10:6379?password=my_secert_password&db=8`. We only support `redis` and API compatible alternative.
Once that done, restart the container.
---
### There are weirdly named directories in my data path?
Unfortunately, That was due to a bug introduced in (2023-09-12 877a41a) and was fixed in (2023-09-19 a2f8c8a), if you
have happened to installation or update during this period, you will have those directories. To fix this issue, you
can simply delete those folders `%(tmpDir)` `%(path)` `{path}` `{tmpDir}`. I decided to not do it automatically to avoid
any data loss. you should check the directories to make sure they are empty. if not copy the directories to the correct
location and delete the empty directories.
---
### How to get WatchState working with YouTube content/library?
Due to the nature on how people name their youtube files i had to pick something specific for it to work cross supported
@@ -598,7 +585,7 @@ If you having problem adding a backend to `WatchState`, it most likely network r
isn't able to communicate with the media backend. Thus, you will get errors. To make sure the container is able to
communicate with the media backend, you can run the following command and check the output.
If the command fails for any reason, then you most likely have network related problem.
If the command fails for any reason, then you most likely have network related problem or invalid apikey/token.
#### For Plex.
@@ -637,8 +624,7 @@ If everything is working correctly you should see something like this previous j
### I keep receiving this warning in log `INFO: Ignoring [xxx] Episode range, and treating it as single episode. Backend says it covers [00-00]`?
We recently added guard clause to prevent backends from sending possibly invalid episode ranges, as such if you see
this,
We recently added guard clause to prevent backends from sending possibly invalid episode ranges, as such if you see this,
this likely means your backend mis-identified episodes range. By default, we allow an episode to cover up to 4 episodes.
If this is not enough for your library content. fear not we have you covered you can increase the limit by running the
@@ -668,9 +654,13 @@ the webhooks section to enable it.
### Bare metal installation
We officially only support the docker container, however for the brave souls who want to install the tool directly on their server,
You can follow these steps.
#### Requirements
* [PHP](http://https://www.php.net/downloads.php) 8.3+ with fpm installed. with the following extensions `pdo`, `pdo-sqlite`, `mbstring`, `json`, `ctype`, `curl`, `redis`, `sodium` and `simplexml`
* [PHP 8.3](http://https://www.php.net/downloads.php) with both the `CLI` and `fpm` mode.
* PHP Extensions `pdo`, `pdo-sqlite`, `mbstring`, `json`, `ctype`, `curl`, `redis`, `sodium` and `simplexml`.
* [Composer](https://getcomposer.org/download/) for dependency management.
* [Redis-server](https://redis.io/) for caching or a compatible implementation that works with [php-redis](https://github.com/phpredis/phpredis).
* [Caddy](https://caddyserver.com/) for frontend handling. However, you can use whatever you like. As long as it has support for fastcgi.
@@ -690,8 +680,8 @@ $ cd watchstate
$ composer install --no-dev
```
3. Create `.env` inside `./var/config/` if you need to change any of the environment variables refer to[Tool specific environment variables](#tool-specific-environment-variables) for more information. For example,
if you `redis` server is not on the same server or requires a password you can add the following to the `.env` file.
3. Create `.env` inside `./var/config/` if you need to change any of the environment variables refer to [Tool specific environment variables](#tool-specific-environment-variables) for more information. For example,
if your `redis` server is not on the same server or requires a password you can add the following to the `.env` file.
```dotenv
WS_CACHE_URL="redis://127.0.0.1:6379?password=your_password"

8
src/API/Backends/View.php
View File

@@ -14,17 +14,17 @@ final class View
{
use APITraits;
#[Get(Index::URL . '/{id:backend}[/]', name: 'backends.view')]
#[Get(Index::URL . '/{name:backend}[/]', name: 'backends.view')]
public function backendsView(iRequest $request, array $args = []): iResponse
{
if (null === ($id = ag($args, 'id'))) {
if (null === ($name = ag($args, 'name'))) {
return api_error('Invalid value for id path parameter.', HTTP_STATUS::HTTP_BAD_REQUEST);
}
$data = $this->getBackends(name: $id);
$data = $this->getBackends(name: $name);
if (empty($data)) {
return api_error(r("Backend '{backend}' not found.", ['backend ' => $id]), HTTP_STATUS::HTTP_NOT_FOUND);
return api_error(r("Backend '{name}' not found.", ['name' => $name]), HTTP_STATUS::HTTP_NOT_FOUND);
}
$apiUrl = $request->getUri()->withHost('')->withPort(0)->withScheme('');

12
src/API/Webhooks/Index.php → src/API/Backends/Webhooks.php
View File

@@ -2,7 +2,7 @@
declare(strict_types=1);
namespace App\API\Webhooks;
namespace App\API\Backends;
use App\Libs\Attributes\Route\Route;
use App\Libs\Config;
@@ -20,18 +20,16 @@ use Monolog\Logger;
use Psr\Http\Message\ResponseInterface as iResponse;
use Psr\Http\Message\ServerRequestInterface as iRequest;
use Psr\Log\LoggerInterface as iLogger;
use Psr\SimpleCache\CacheInterface;
use Psr\SimpleCache\CacheInterface as iCache;
use Psr\SimpleCache\InvalidArgumentException;
final class Index
final class Webhooks
{
use APITraits;
public const string URL = '%{api.prefix}/webhooks';
private iLogger $accesslog;
public function __construct(private CacheInterface $cache)
public function __construct(private iCache $cache)
{
$this->accesslog = new Logger(name: 'http', processors: [new LogMessageProcessor()]);
@@ -55,7 +53,7 @@ final class Index
* @return iResponse The response object.
* @throws InvalidArgumentException if cache key is invalid.
*/
#[Route(['POST', 'PUT'], self::URL . '/{name:backend}[/]', name: 'webhooks.receive')]
#[Route(['POST', 'PUT'], Index::URL . '/{name:backend}/webhook[/]', name: 'webhooks.receive')]
public function __invoke(iRequest $request, array $args = []): iResponse
{
if (null === ($name = ag($args, 'name'))) {

245
src/Libs/Initializer.php
View File

@@ -4,16 +4,14 @@ declare(strict_types=1);
namespace App\Libs;
use App\API\Backends\Webhooks;
use App\Cli;
use App\Libs\Entity\StateInterface as iState;
use App\Libs\Exceptions\Backends\RuntimeException;
use App\Libs\Exceptions\HttpException;
use App\Libs\Exceptions\InvalidArgumentException;
use App\Libs\Extends\ConsoleHandler;
use App\Libs\Extends\ConsoleOutput;
use App\Libs\Extends\RouterStrategy;
use Closure;
use DateInterval;
use ErrorException;
use League\Route\Http\Exception as RouterHttpException;
use League\Route\RouteGroup;
@@ -179,7 +177,7 @@ final class Initializer
}
/**
* Run the application in HTTP Context.
* Run the application in HTTP context.
*
* @param iRequest|null $request If null, the request will be created from globals.
* @param callable(iResponse):void|null $emitter If null, the emitter will be created from globals.
@@ -229,20 +227,18 @@ final class Initializer
}
/**
* Handle HTTP requests and process webhooks.
* Proxy API requests to the API server, and handle old style webhooks.
* into the new API server.
*
* @param iRequest $realRequest The incoming HTTP request.
* @param iRequest $request The incoming HTTP request.
*
* @return iResponse The HTTP response.
*
* @throws \Psr\SimpleCache\InvalidArgumentException If cache key is illegal.
*/
private function defaultHttpServer(iRequest $realRequest): iResponse
private function defaultHttpServer(iRequest $request): iResponse
{
$log = $backend = [];
$class = null;
$backend = [];
$request = $realRequest;
$requestPath = $request->getUri()->getPath();
// -- health endpoint.
@@ -260,30 +256,21 @@ final class Initializer
// -- Forward requests to API server.
if (true === str_starts_with($requestPath, Config::get('api.prefix', '????'))) {
return $this->defaultAPIServer(clone $realRequest);
return $this->defaultAPIServer(clone $request);
}
// -- Save request payload.
if (true === Config::get('webhook.dumpRequest')) {
saveRequestPayload(clone $realRequest);
}
$apikey = ag($realRequest->getQueryParams(), 'apikey', $realRequest->getHeaderLine('x-apikey'));
$apikey = ag($request->getQueryParams(), 'apikey', $request->getHeaderLine('x-apikey'));
if (empty($apikey)) {
$response = api_response(HTTP_STATUS::HTTP_UNAUTHORIZED);
$this->write(
$request,
Level::Info,
$this->formatLog($request, $response, 'No webhook token was found in header or query.')
);
$this->write($request, Level::Info, $this->formatLog($request, $response, 'No webhook token was found.'));
return $response;
}
$validUser = $validUUid = null;
$configFile = ConfigFile::open(Config::get('backends_file'), 'yaml');
// -- Find Relevant backend.
foreach (Config::get('servers', []) as $name => $info) {
foreach ($configFile->getAll() as $name => $info) {
if (null === ag($info, 'webhook.token')) {
continue;
}
@@ -292,213 +279,25 @@ final class Initializer
continue;
}
try {
$class = makeBackend($info, $name);
} catch (InvalidArgumentException $e) {
$this->write(
request: $request,
level: Level::Error,
message: 'Exception [{error.kind}] was thrown unhandled in [{backend}] instance creation. Error [{error.message} @ {error.file}:{error.line}].',
context: [
'backend' => $name,
'error' => [
'kind' => $e::class,
'line' => $e->getLine(),
'message' => $e->getMessage(),
'file' => after($e->getFile(), ROOT_PATH),
],
'exception' => [
'file' => $e->getFile(),
'line' => $e->getLine(),
'kind' => get_class($e),
'message' => $e->getMessage(),
'trace' => $e->getTrace(),
]
]
);
continue;
}
$request = $class->processRequest(clone $realRequest);
$attr = $request->getAttributes();
if (null !== ($userId = ag($info, 'user', null)) && true === (bool)ag($info, 'webhook.match.user')) {
if (null === ($requestUser = ag($attr, 'user.id'))) {
$validUser = false;
$backend = $class = null;
$log[] = 'Request user is not set';
continue;
}
if (false === hash_equals((string)$userId, (string)$requestUser)) {
$validUser = false;
$backend = $class = null;
$log[] = r('Request user id [{req_user}] does not match configured value [{config_user}]', [
'req_user' => $requestUser ?? 'NOT SET',
'config_user' => $userId,
]);
continue;
}
$validUser = true;
}
if (null !== ($uuid = ag($info, 'uuid', null)) && true === (bool)ag($info, 'webhook.match.uuid')) {
if (null === ($requestBackendId = ag($attr, 'backend.id'))) {
$validUUid = false;
$backend = $class = null;
$log[] = 'backend unique id is not set';
continue;
}
if (false === hash_equals((string)$uuid, (string)$requestBackendId)) {
$validUUid = false;
$backend = $class = null;
$log[] = r('Request backend unique id [{req_uid}] does not match backend uuid [{config_uid}].', [
'req_uid' => $requestBackendId ?? 'NOT SET',
'config_uid' => $uuid,
]);
continue;
}
$validUUid = true;
}
$backend = array_replace_recursive(['name' => $name], $info);
$info['name'] = $name;
$backend = $info;
break;
}
if (empty($backend) || null === $class) {
if (false === $validUser) {
$loglevel = Level::Debug;
$message = 'token is valid, User matching failed.';
} elseif (false === $validUUid) {
$message = 'token and user are valid. Backend unique id matching failed.';
} else {
$message = 'Invalid token was given.';
}
if (empty($backend)) {
$response = api_response(HTTP_STATUS::HTTP_UNAUTHORIZED);
$this->write(
$request,
$loglevel ?? Level::Error,
$this->formatLog($request, $response, $message),
['messages' => $log, 'attr' => $attr ?? []],
forceContext: true
);
$this->write($request, Level::Info, $this->formatLog($request, $response, 'Invalid token was given.'));
return $response;
}
// -- sanity check in case user has both import.enabled and options.IMPORT_METADATA_ONLY enabled.
if (true === (bool)ag($backend, 'import.enabled')) {
if (true === ag_exists($backend, 'options.' . Options::IMPORT_METADATA_ONLY)) {
$backend = ag_delete($backend, 'options.' . Options::IMPORT_METADATA_ONLY);
}
}
$metadataOnly = true === (bool)ag($backend, 'options.' . Options::IMPORT_METADATA_ONLY);
if (true !== $metadataOnly && true !== (bool)ag($backend, 'import.enabled')) {
$response = api_response(HTTP_STATUS::HTTP_NOT_ACCEPTABLE);
$this->write(
$request,
Level::Error,
$this->formatLog($request, $response, 'Import are disabled for [{backend}].'),
[
'backend' => $class->getName(),
],
forceContext: true
);
return $response;
}
$entity = $class->parseWebhook($request);
// -- Dump Webhook context.
if (true === (bool)ag($backend, 'options.' . Options::DUMP_PAYLOAD)) {
saveWebhookPayload($entity, $request);
}
if (!$entity->hasGuids() && !$entity->hasRelativeGuid()) {
$this->write(
$request,
Level::Info,
'Ignoring [{backend}] {item.type} [{item.title}]. No valid/supported external ids.',
[
'backend' => $entity->via,
'item' => [
'title' => $entity->getName(),
'type' => $entity->type,
],
]
);
return api_response(HTTP_STATUS::HTTP_NOT_MODIFIED);
}
if ((0 === (int)$entity->episode || null === $entity->season) && $entity->isEpisode()) {
$this->write(
$request,
Level::Notice,
'Ignoring [{backend}] {item.type} [{item.title}]. No episode/season number present.',
[
'backend' => $entity->via,
'item' => [
'title' => $entity->getName(),
'type' => $entity->type,
'season' => (string)($entity->season ?? 'None'),
'episode' => (string)($entity->episode ?? 'None'),
]
]
);
return api_response(HTTP_STATUS::HTTP_NOT_MODIFIED);
}
$cache = Container::get(CacheInterface::class);
$items = $cache->get('requests', []);
$itemId = r('{type}://{id}:{tainted}@{backend}', [
'type' => $entity->type,
'backend' => $entity->via,
'tainted' => $entity->isTainted() ? 'tainted' : 'untainted',
'id' => ag($entity->getMetadata($entity->via), iState::COLUMN_ID, '??'),
]);
$items[$itemId] = [
'options' => [
Options::IMPORT_METADATA_ONLY => $metadataOnly,
],
'entity' => $entity,
];
$cache->set('requests', $items, new DateInterval('P3D'));
if (false === $metadataOnly && true === $entity->hasPlayProgress()) {
$progress = $cache->get('progress', []);
$progress[$itemId] = $entity;
$cache->set('progress', $progress, new DateInterval('P1D'));
}
$this->write($request, Level::Info, 'Queued [{backend}: {event}] {item.type} [{item.title}].', [
'backend' => $entity->via,
'event' => ag($entity->getExtra($entity->via), iState::COLUMN_EXTRA_EVENT),
'has_progress' => $entity->hasPlayProgress() ? 'Yes' : 'No',
'item' => [
'title' => $entity->getName(),
'type' => $entity->type,
'played' => $entity->isWatched() ? 'Yes' : 'No',
'queue_id' => $itemId,
'progress' => $entity->hasPlayProgress() ? $entity->getPlayProgress() : null,
]
$uri = r('/v1/api/backends/{backend}/webhook', ['backend' => ag($backend, 'name')]);
return Container::get(Webhooks::class)(
$request->withUri($request->getUri()->withPath($uri)->withQuery(''))
->withHeader('Authorization', 'Bearer ' . Config::get('api.key'))->withoutHeader('X-Apikey'),
[
'name' => $backend['name']
]
);
return api_response(HTTP_STATUS::HTTP_OK);
}
/**