Added File integrity feature.
This commit is contained in:
17
FAQ.md
17
FAQ.md
@@ -867,3 +867,20 @@ $ ./bin/console help
|
|||||||
```
|
```
|
||||||
|
|
||||||
For more information, please refer to the [Dockerfile](/Dockerfile). On how we do things to get the tool running.
|
For more information, please refer to the [Dockerfile](/Dockerfile). On how we do things to get the tool running.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### How does the file integrity feature works?
|
||||||
|
|
||||||
|
The feature first scan your entire history for reported media file paths. then we will do stat check on each point of the path starting from lowest to highest.
|
||||||
|
|
||||||
|
For example lets say your media file is `/media/series/season 1/episode 1.mkv`
|
||||||
|
|
||||||
|
We do the following. `/media` exists or not? if it does we move to the next path `/media/series` exists or not? if it does we move to the next path `/media/series/season 1` exists or not? if it does we move to the next path `/media/series/season 1/episode 1.mkv` exists or not? if it does we move to the next path.
|
||||||
|
|
||||||
|
Using this approach allow us to cache calls and reduce unnecessary calls to the filesystem. If you have for example `/media/seriesX/` with thousands of files,
|
||||||
|
and the root `/media/seriesX` doesn't exists we dont have to call stat for every file, instead using the cache we determine that the file doesn't exist.
|
||||||
|
|
||||||
|
Of course, every stat call is cached, so if 1 or more backends are reporting the same file path, we only do the stat check once. This is to reduce the load on the filesystem.
|
||||||
|
|
||||||
|
---
|
||||||
|
|||||||
38
NEWS.md
38
NEWS.md
@@ -1,5 +1,43 @@
|
|||||||
# Old Updates
|
# Old Updates
|
||||||
|
|
||||||
|
### 2024-07-06
|
||||||
|
|
||||||
|
Recently we have introduced a new feature that allows you to use Jellyfin and Emby OAuth access tokens for syncing
|
||||||
|
your play state. This is especially handy if you're not the server owner and can't create API keys. Please note, this
|
||||||
|
feature is in its experimental phase, so you might encounter some issues as we yet to explorer the full depth of the
|
||||||
|
implementation. We're actively working on making it better, If you have any feedback or suggestions, please let us know.
|
||||||
|
|
||||||
|
Getting your OAuth token is easy. When prompted, simply enter your `username:password` in place of the API key through
|
||||||
|
the `WebUI` or the `config:add/manage` command. `WatchState` will automatically contact the backend and generate the
|
||||||
|
token for you, as this step is required to get more information like your `User ID` which is sadly inaccessible without
|
||||||
|
us generating the token. Both Emby & Jellyfin doesn't provide an API endpoint to inquiry about the current user.
|
||||||
|
|
||||||
|
We have also added new `config:test` command to run functional tests on your backends, this will not alter your state,
|
||||||
|
And it's quite useful to know if the tool is able to communicate with your backends. without problems, It will report
|
||||||
|
the following, `OK` which mean the indicated test has passed, `FA` which mean the indicated test has failed. And `SK`
|
||||||
|
which mean the indicated test has been skipped or not yet implemented.
|
||||||
|
|
||||||
|
### 2024-06-23
|
||||||
|
|
||||||
|
WE are happy to announce that the `WebUI` is ready for wider usage and we are planning to release it in the next few
|
||||||
|
months.
|
||||||
|
We are actively working on it to improve it. If you have any feedback or suggestions, please let us know. We feel it's
|
||||||
|
almost future complete
|
||||||
|
for the things that we want.
|
||||||
|
|
||||||
|
On another related news, we have added new environment variable `WS_API_AUTO` "disabled by default" which can be used
|
||||||
|
to automatically expose your **API KEY/TOKEN**. This is useful for users who are using the `WebUI` from many different
|
||||||
|
browsers
|
||||||
|
and want to automate the configuration process.
|
||||||
|
|
||||||
|
While the `WebUI` is included in the main project, it's a standalone feature and requires the API settings to be
|
||||||
|
configured before it
|
||||||
|
can be used. This environment variable can be enabled by setting `WS_API_AUTO=true` in `${WS_DATA_PATH}/config/.env`.
|
||||||
|
|
||||||
|
> [!IMPORTANT]
|
||||||
|
> This environment variable is **GREAT SECURITY RISK**, and we strongly recommend not to use it if `WatchState` is
|
||||||
|
> exposed to the internet.
|
||||||
|
|
||||||
### 2024-05-14
|
### 2024-05-14
|
||||||
|
|
||||||
We are happy to announce the beta testing of the `WebUI`. To get started on using it you just need to visit the url `http://localhost:8080` We are supposed to
|
We are happy to announce the beta testing of the `WebUI`. To get started on using it you just need to visit the url `http://localhost:8080` We are supposed to
|
||||||
|
|||||||
43
README.md
43
README.md
@@ -9,43 +9,20 @@ out of the box, this tool support `Jellyfin`, `Plex` and `Emby` media servers.
|
|||||||
|
|
||||||
## Updates
|
## Updates
|
||||||
|
|
||||||
### 2024-07-06
|
### 2024-07-22
|
||||||
|
|
||||||
Recently we have introduced a new feature that allows you to use Jellyfin and Emby OAuth access tokens for syncing
|
We have recently added a new WebUI feature, `File integrity`, this feature will help you to check if your media backends
|
||||||
your play state. This is especially handy if you're not the server owner and can't create API keys. Please note, this
|
are reporting files that are not available on the disk. This feature is still in alpha, and we are working on improving
|
||||||
feature is in its experimental phase, so you might encounter some issues as we yet to explorer the full depth of the
|
it.
|
||||||
implementation. We're actively working on making it better, If you have any feedback or suggestions, please let us know.
|
|
||||||
|
|
||||||
Getting your OAuth token is easy. When prompted, simply enter your `username:password` in place of the API key through
|
This feature `REQUIRES` that you mount your media directories to the `WatchState` container preferably as readonly. There is plans to add
|
||||||
the `WebUI` or the `config:add/manage` command. `WatchState` will automatically contact the backend and generate the
|
a path replacement feature to allow you change the pathing, but it's not implemented yet.
|
||||||
token for you, as this step is required to get more information like your `User ID` which is sadly inaccessible without
|
|
||||||
us generating the token. Both Emby & Jellyfin doesn't provide an API endpoint to inquiry about the current user.
|
|
||||||
|
|
||||||
We have also added new `config:test` command to run functional tests on your backends, this will not alter your state,
|
This feature will work on both local and remote cloud storages provided they are mounted into the container. We also may recommend not to
|
||||||
And it's quite useful to know if the tool is able to communicate with your backends. without problems, It will report
|
use this feature depending on how your cloud storage provider treats file stat calls. As it might lead to unnecessary money spending. and of course
|
||||||
the following, `OK` which mean the indicated test has passed, `FA` which mean the indicated test has failed. And `SK`
|
it will be slower.
|
||||||
which mean the indicated test has been skipped or not yet implemented.
|
|
||||||
|
|
||||||
### 2024-06-23
|
For more information about how we cache the stat calls, please refer to the [FAQ](FAQ.md#How-does-the-file-integrity-feature-works).
|
||||||
|
|
||||||
WE are happy to announce that the `WebUI` is ready for wider usage and we are planning to release it in the next few
|
|
||||||
months.
|
|
||||||
We are actively working on it to improve it. If you have any feedback or suggestions, please let us know. We feel it's
|
|
||||||
almost future complete
|
|
||||||
for the things that we want.
|
|
||||||
|
|
||||||
On another related news, we have added new environment variable `WS_API_AUTO` "disabled by default" which can be used
|
|
||||||
to automatically expose your **API KEY/TOKEN**. This is useful for users who are using the `WebUI` from many different
|
|
||||||
browsers
|
|
||||||
and want to automate the configuration process.
|
|
||||||
|
|
||||||
While the `WebUI` is included in the main project, it's a standalone feature and requires the API settings to be
|
|
||||||
configured before it
|
|
||||||
can be used. This environment variable can be enabled by setting `WS_API_AUTO=true` in `${WS_DATA_PATH}/config/.env`.
|
|
||||||
|
|
||||||
> [!IMPORTANT]
|
|
||||||
> This environment variable is **GREAT SECURITY RISK**, and we strongly recommend not to use it if `WatchState` is
|
|
||||||
> exposed to the internet.
|
|
||||||
|
|
||||||
Refer to [NEWS](NEWS.md) for old updates.
|
Refer to [NEWS](NEWS.md) for old updates.
|
||||||
|
|
||||||
|
|||||||
@@ -68,6 +68,11 @@
|
|||||||
<span class="icon"><i class="fas fa-database"></i></span>
|
<span class="icon"><i class="fas fa-database"></i></span>
|
||||||
<span>Data Parity</span>
|
<span>Data Parity</span>
|
||||||
</NuxtLink>
|
</NuxtLink>
|
||||||
|
|
||||||
|
<NuxtLink class="navbar-item" to="/integrity" @click.native="showMenu=false">
|
||||||
|
<span class="icon"><i class="fas fa-file"></i></span>
|
||||||
|
<span>Files Integrity</span>
|
||||||
|
</NuxtLink>
|
||||||
<hr class="navbar-divider">
|
<hr class="navbar-divider">
|
||||||
|
|
||||||
<NuxtLink class="navbar-item" to="/events" @click.native="showMenu=false">
|
<NuxtLink class="navbar-item" to="/events" @click.native="showMenu=false">
|
||||||
|
|||||||
386
frontend/pages/integrity.vue
Normal file
386
frontend/pages/integrity.vue
Normal file
@@ -0,0 +1,386 @@
|
|||||||
|
<template>
|
||||||
|
<div>
|
||||||
|
<div class="columns is-multiline">
|
||||||
|
<div class="column is-12 is-clearfix is-unselectable">
|
||||||
|
<span class="title is-4">
|
||||||
|
<span class="icon"><i class="fas fa-file"></i></span>
|
||||||
|
Files Integrity
|
||||||
|
</span>
|
||||||
|
<div class="is-pulled-right" v-if="isLoaded">
|
||||||
|
<div class="field is-grouped">
|
||||||
|
|
||||||
|
<p class="control" v-if="isCached">
|
||||||
|
<button class="button is-danger" @click="emptyCache" v-tooltip.bottom="'Empty cache.'"
|
||||||
|
:disabled="isDeleting || isLoading">
|
||||||
|
<span class="icon"><i class="fas fa-box-archive"></i></span>
|
||||||
|
</button>
|
||||||
|
</p>
|
||||||
|
|
||||||
|
<div class="control has-icons-left" v-if="showFilter">
|
||||||
|
<input type="search" v-model.lazy="filter" class="input" id="filter"
|
||||||
|
placeholder="Filter displayed results.">
|
||||||
|
<span class="icon is-left">
|
||||||
|
<i class="fas fa-filter"></i>
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="control">
|
||||||
|
<button class="button is-danger is-light" @click="toggleFilter">
|
||||||
|
<span class="icon"><i class="fas fa-filter"></i></span>
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<p class="control">
|
||||||
|
<button class="button is-danger" @click="massDelete" v-tooltip.bottom="'Delete selected records.'"
|
||||||
|
:disabled="isDeleting || isLoading || selected_ids.length<1"
|
||||||
|
:class="{'is-loading':isDeleting}">
|
||||||
|
<span class="icon"><i class="fas fa-trash"></i></span>
|
||||||
|
</button>
|
||||||
|
</p>
|
||||||
|
|
||||||
|
<div class="control">
|
||||||
|
<button class="button is-info is-light" @click="selectAll = !selectAll"
|
||||||
|
data-tooltip="Toggle select all">
|
||||||
|
<span class="icon">
|
||||||
|
<i class="fas fa-check-square"
|
||||||
|
:class="{ 'fa-check-square': !selectAll, 'fa-square':selectAll}"></i>
|
||||||
|
</span>
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<p class="control">
|
||||||
|
<button class="button is-info" @click.prevent="loadContent()" :disabled="isLoading"
|
||||||
|
:class="{'is-loading':isLoading}">
|
||||||
|
<span class="icon"><i class="fas fa-sync"></i></span>
|
||||||
|
</button>
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="is-hidden-mobile">
|
||||||
|
<span class="subtitle">This page will show records with files that no longer exist on the system.</span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="column is-12" v-if="isLoaded">
|
||||||
|
<Message v-if="isLoading" message_class="has-background-info-90 has-text-dark" title="Loading"
|
||||||
|
icon="fas fa-spinner fa-spin" message="Loading data. Please wait..."/>
|
||||||
|
|
||||||
|
<Message message_class="has-background-warning-80 has-text-dark" v-if="filter && filteredRows(items).length < 1"
|
||||||
|
title="Information"
|
||||||
|
icon="fas fa-check">
|
||||||
|
The filter <code>{{ filter }}</code> did not match any records.
|
||||||
|
</Message>
|
||||||
|
<Message message_class="has-background-success-90 has-text-dark" v-else title="Success" icon="fas fa-check"
|
||||||
|
v-if="!isLoading && items.length<1">
|
||||||
|
WatchState did not find any file references that are no longer on the system.
|
||||||
|
</Message>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="columns is-multiline" v-if="!isLoaded">
|
||||||
|
<div class="column is-12">
|
||||||
|
<div class="card">
|
||||||
|
<header class="card-header">
|
||||||
|
<p class="card-header-title is-justify-center">Request File integrity check.</p>
|
||||||
|
</header>
|
||||||
|
<div class="card-content">
|
||||||
|
<div class="content">
|
||||||
|
<ul>
|
||||||
|
<li>
|
||||||
|
Please be aware, this process will take time. You will see the spinner while <code>WatchState</code>
|
||||||
|
is analyzing the entire history records. Do not reload the page.
|
||||||
|
</li>
|
||||||
|
<li>This check <strong><code>REQUIRES</code></strong> that the file contents be accessible to
|
||||||
|
<code>WatchState</code>. You should mount your library in <code>compose.yml</code> file as readonly.
|
||||||
|
<span class="is-bold">If you do not mount your library. every record will fail the check.</span>
|
||||||
|
</li>
|
||||||
|
<li>There are no path replacement support at the moment. The pathing must match what your media servers
|
||||||
|
are reporting. There are plans to add this feature in the future.
|
||||||
|
</li>
|
||||||
|
<li>This process will do two checks, One will do dir stat on the file directory, and file stat on the
|
||||||
|
file itself if the directory exists. <span class="has-text-danger">If you are using cloud storage, we
|
||||||
|
recommend to not use this check. as it will be slow. and probably will cost you a lot of
|
||||||
|
money.</span>
|
||||||
|
</li>
|
||||||
|
<li>The process caches the file and dir stat, as such we only run stat once per file or directory no
|
||||||
|
matter how many backends reports the same path or file.
|
||||||
|
</li>
|
||||||
|
<li>The results are cached server side for one hour from the request.</li>
|
||||||
|
</ul>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="control">
|
||||||
|
<button class="button is-fullwidth is-primary" @click="loadContent" :disabled="isLoading">
|
||||||
|
<span class="icon"><i class="fas fa-check"></i></span>
|
||||||
|
<span>Initiate The process</span>
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="columns is-multiline" v-if="!isLoading && isLoaded">
|
||||||
|
<div class="column is-12">
|
||||||
|
<div class="columns is-multiline" v-if="filteredRows(items)?.length>0">
|
||||||
|
<template v-for="item in items" :key="item.id">
|
||||||
|
<Lazy :unrender="true" :min-height="343" class="column is-6-tablet" v-if="filterItem(item)">
|
||||||
|
<div class="card" :class="{ 'is-success': item.watched }">
|
||||||
|
<header class="card-header">
|
||||||
|
<p class="card-header-title is-text-overflow pr-1">
|
||||||
|
<span class="icon">
|
||||||
|
<label class="checkbox">
|
||||||
|
<input type="checkbox" :value="item.id" v-model="selected_ids">
|
||||||
|
</label>
|
||||||
|
</span>
|
||||||
|
<NuxtLink :to="'/history/'+item.id" v-text="makeName(item)"/>
|
||||||
|
</p>
|
||||||
|
<span class="card-header-icon" @click="item.showRawData = !item?.showRawData">
|
||||||
|
<span class="icon">
|
||||||
|
<i class="fas"
|
||||||
|
:class="{ 'fa-tv': 'episode' === item.type.toLowerCase(), 'fa-film': 'movie' === item.type.toLowerCase()}"></i>
|
||||||
|
</span>
|
||||||
|
</span>
|
||||||
|
</header>
|
||||||
|
<div class="card-content">
|
||||||
|
<div class="columns is-multiline is-mobile">
|
||||||
|
<div class="column is-12">
|
||||||
|
<div class="field is-grouped">
|
||||||
|
<div class="control is-clickable"
|
||||||
|
:class="{'is-text-overflow': !item?.expand_title, 'is-text-contents': item?.expand_title}"
|
||||||
|
@click="item.expand_title = !item?.expand_title">
|
||||||
|
<span class="icon"><i class="fas fa-heading"></i> </span>
|
||||||
|
<template v-if="item?.content_title">
|
||||||
|
<NuxtLink :to="makeSearchLink('subtitle', item.content_title)" v-text="item.content_title"/>
|
||||||
|
</template>
|
||||||
|
<template v-else>
|
||||||
|
<NuxtLink :to="makeSearchLink('subtitle', item.title)" v-text="item.title"/>
|
||||||
|
</template>
|
||||||
|
</div>
|
||||||
|
<div class="control">
|
||||||
|
<span class="icon is-clickable"
|
||||||
|
@click="copyText(item?.content_title ?? item.title, false)">
|
||||||
|
<i class="fas fa-copy"></i></span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="column is-12">
|
||||||
|
<div class="field is-grouped">
|
||||||
|
<div class="control is-clickable"
|
||||||
|
:class="{'is-text-overflow': !item?.expand_path, 'is-text-contents': item?.expand_path}"
|
||||||
|
@click="item.expand_path = !item?.expand_path">
|
||||||
|
<span class="icon"><i class="fas fa-file"></i> </span>
|
||||||
|
<NuxtLink v-if="item?.content_path" :to="makeSearchLink('path', item.content_path)"
|
||||||
|
v-text="item.content_path"/>
|
||||||
|
<span v-else>No path found.</span>
|
||||||
|
</div>
|
||||||
|
<div class="control">
|
||||||
|
<span class="icon is-clickable"
|
||||||
|
@click="copyText(item?.content_path ?item.content_path : null, false)">
|
||||||
|
<i class="fas fa-copy"></i></span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="column is-12">
|
||||||
|
<div class="field is-grouped">
|
||||||
|
<div class="control is-expanded is-unselectable">
|
||||||
|
<span class="icon"><i class="fas fa-info"></i> </span>
|
||||||
|
<span>Has metadata from</span>
|
||||||
|
</div>
|
||||||
|
<div class="control">
|
||||||
|
<template v-for="backend in item.reported_by" :key="`${item.id}-rb-${backend}`">
|
||||||
|
<NuxtLink :to="'/backend/'+backend" v-text="backend" class="tag is-primary ml-1"/>
|
||||||
|
</template>
|
||||||
|
<template v-for="backend in item.not_reported_by" :key="`${item.id}-rb-${backend}`">
|
||||||
|
<NuxtLink :to="'/backend/'+backend" v-text="backend" class="tag is-danger ml-1"/>
|
||||||
|
</template>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="column is-12" v-if="item?.integrity">
|
||||||
|
<template v-for="record in item.integrity" :key="`integrity-${record.backend}`">
|
||||||
|
<p>
|
||||||
|
<span class="icon">
|
||||||
|
<i class="fas" :class="{'fa-exclamation-triangle':!item.status,'fa-check':item.status}"></i>
|
||||||
|
</span>
|
||||||
|
<span :class="{'has-text-danger':!item.status,'has-text-success':item.status}">
|
||||||
|
{{ record.backend }}: {{ record.message }}</span>
|
||||||
|
</p>
|
||||||
|
</template>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="card-content p-0 m-0" v-if="item?.showRawData">
|
||||||
|
<pre style="position: relative; max-height: 343px;"><code>{{ JSON.stringify(item, null, 2) }}</code>
|
||||||
|
<button class="button is-small m-4" @click="() => copyText(JSON.stringify(item, null, 2))"
|
||||||
|
style="position: absolute; top:0; right:0;">
|
||||||
|
<span class="icon"><i class="fas fa-copy"></i></span>
|
||||||
|
</button>
|
||||||
|
</pre>
|
||||||
|
</div>
|
||||||
|
<div class="card-footer">
|
||||||
|
<div class="card-footer-item">
|
||||||
|
<span class="icon">
|
||||||
|
<i class="fas" :class="{'fa-eye':item.watched,'fa-eye-slash':!item.watched}"></i>
|
||||||
|
</span>
|
||||||
|
<span class="has-text-success" v-if="item.watched">Played</span>
|
||||||
|
<span class="has-text-danger" v-else>Unplayed</span>
|
||||||
|
</div>
|
||||||
|
<div class="card-footer-item">
|
||||||
|
<span class="icon"><i class="fas fa-calendar"></i> </span>
|
||||||
|
<span class="has-tooltip"
|
||||||
|
v-tooltip="`Record updated at: ${moment.unix(item.updated_at).format(TOOLTIP_DATE_FORMAT)}`">
|
||||||
|
{{ moment.unix(item.updated_at).fromNow() }}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</lazy>
|
||||||
|
</template>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</template>
|
||||||
|
|
||||||
|
<script setup>
|
||||||
|
import request from '~/utils/request'
|
||||||
|
import Message from '~/components/Message'
|
||||||
|
import {awaitElement, copyText, makeName, makeSearchLink, notification, TOOLTIP_DATE_FORMAT} from '~/utils/index'
|
||||||
|
import moment from 'moment'
|
||||||
|
import Lazy from '~/components/Lazy'
|
||||||
|
|
||||||
|
useHead({title: 'File Integrity'})
|
||||||
|
|
||||||
|
const items = ref([])
|
||||||
|
const isLoading = ref(false)
|
||||||
|
const isLoaded = ref(false)
|
||||||
|
const selected_ids = ref([])
|
||||||
|
const isDeleting = ref(false)
|
||||||
|
const filter = ref('')
|
||||||
|
const showFilter = ref(false)
|
||||||
|
const isCached = ref(false)
|
||||||
|
|
||||||
|
const selectAll = ref(false)
|
||||||
|
const massActionInProgress = ref(false)
|
||||||
|
watch(selectAll, v => selected_ids.value = v ? filteredRows(items.value).map(i => i.id) : []);
|
||||||
|
|
||||||
|
const toggleFilter = () => {
|
||||||
|
showFilter.value = !showFilter.value
|
||||||
|
if (!showFilter.value) {
|
||||||
|
filter.value = ''
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
awaitElement('#filter', (_, elm) => elm.focus())
|
||||||
|
}
|
||||||
|
|
||||||
|
const loadContent = async () => {
|
||||||
|
isLoaded.value = true
|
||||||
|
isLoading.value = true
|
||||||
|
items.value = []
|
||||||
|
selectAll.value = false
|
||||||
|
selected_ids.value = []
|
||||||
|
|
||||||
|
try {
|
||||||
|
const response = await request(`/system/integrity`)
|
||||||
|
const json = await response.json()
|
||||||
|
|
||||||
|
if (200 !== response.status) {
|
||||||
|
notification('error', 'Error', `API Error. ${json.error.code}: ${json.error.message}`)
|
||||||
|
isLoading.value = false
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if (json.items) {
|
||||||
|
items.value = json.items
|
||||||
|
}
|
||||||
|
|
||||||
|
isLoading.value = false
|
||||||
|
isCached.value = Boolean(json?.fromCache ?? false)
|
||||||
|
} catch (e) {
|
||||||
|
notification('error', 'Error', `Request error. ${e.message}`)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const massDelete = async () => {
|
||||||
|
if (0 === selected_ids.value.length) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!confirm(`Are you sure you want to delete '${selected_ids.value.length}' item/s?`)) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
isDeleting.value = true
|
||||||
|
|
||||||
|
const urls = selected_ids.value.map(id => `/history/${id}`)
|
||||||
|
|
||||||
|
notification('success', 'Action in progress', `Deleting '${urls.length}' item/s. Please wait...`)
|
||||||
|
|
||||||
|
// -- check each request response after all requests are done
|
||||||
|
const requests = await Promise.all(urls.map(url => request(url, {method: 'DELETE'})))
|
||||||
|
|
||||||
|
if (!requests.every(response => 200 === response.status)) {
|
||||||
|
notification('error', 'Error', `Some requests failed. Please check the console for more details.`)
|
||||||
|
} else {
|
||||||
|
items.value = items.value.filter(i => !selected_ids.value.includes(i.id))
|
||||||
|
}
|
||||||
|
|
||||||
|
notification('success', 'Success', `Deleting '${urls.length}' item/s completed.`)
|
||||||
|
} catch (e) {
|
||||||
|
notification('error', 'Error', `Request error. ${e.message}`)
|
||||||
|
} finally {
|
||||||
|
massActionInProgress.value = false
|
||||||
|
selected_ids.value = []
|
||||||
|
selectAll.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const emptyCache = async () => {
|
||||||
|
if (!confirm(`Are you sure you want to purge file stats cache?`)) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
|
||||||
|
const response = await request(`/system/integrity`, {method: 'DELETE'})
|
||||||
|
if (200 !== response.status) {
|
||||||
|
const json = await response.json()
|
||||||
|
return notification('error', 'Error', `API Error. ${json.error.code}: ${json.error.message}`)
|
||||||
|
}
|
||||||
|
|
||||||
|
items.value = [];
|
||||||
|
isLoaded.value = false
|
||||||
|
isLoading.value = false
|
||||||
|
isCached.value = false
|
||||||
|
selectAll.value = false
|
||||||
|
selected_ids.value = []
|
||||||
|
|
||||||
|
notification('success', 'Success', `Cache purged.`)
|
||||||
|
} catch (e) {
|
||||||
|
notification('error', 'Error', `Request error. ${e.message}`)
|
||||||
|
} finally {
|
||||||
|
massActionInProgress.value = false
|
||||||
|
selected_ids.value = []
|
||||||
|
selectAll.value = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const filteredRows = items => {
|
||||||
|
if (!filter.value) {
|
||||||
|
return items
|
||||||
|
}
|
||||||
|
|
||||||
|
return items.filter(i => Object.values(i).some(v => typeof v === 'string' ? v.toLowerCase().includes(filter.value.toLowerCase()) : false))
|
||||||
|
}
|
||||||
|
|
||||||
|
const filterItem = item => {
|
||||||
|
if (!filter.value || !item) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
return Object.values(item).some(v => typeof v === 'string' ? v.toLowerCase().includes(filter.value.toLowerCase()) : false)
|
||||||
|
}
|
||||||
|
</script>
|
||||||
@@ -174,29 +174,14 @@
|
|||||||
<div class="column is-12">
|
<div class="column is-12">
|
||||||
<div class="field is-grouped">
|
<div class="field is-grouped">
|
||||||
<div class="control is-expanded is-unselectable">
|
<div class="control is-expanded is-unselectable">
|
||||||
<span class="icon"><i class="fas fa-check"></i> </span>
|
<span class="icon"><i class="fas fa-info"></i> </span>
|
||||||
<span>Reported By</span>
|
<span>Has metadata from</span>
|
||||||
</div>
|
</div>
|
||||||
<div class="control">
|
<div class="control">
|
||||||
<template v-for="backend in item.reported_by" :key="`${item.id}-rb-${backend}`">
|
<NuxtLink v-for="backend in item.reported_by" :key="`${item.id}-rb-${backend}`"
|
||||||
<NuxtLink :to="'/backend/'+backend" v-text="backend" class="tag"/>
|
:to="'/backend/'+backend" v-text="backend" class="tag is-primary ml-1"/>
|
||||||
|
<NuxtLink v-for="backend in item.not_reported_by" :key="`${item.id}-nrb-${backend}`"
|
||||||
</template>
|
:to="'/backend/'+backend" v-text="backend" class="tag is-danger ml-1"/>
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="column is-12">
|
|
||||||
<div class="field is-grouped">
|
|
||||||
<div class="control is-expanded is-unselectable">
|
|
||||||
<span class="icon"><i class="fas fa-times"></i> </span>
|
|
||||||
<span>Not Reported By</span>
|
|
||||||
</div>
|
|
||||||
<div class="control">
|
|
||||||
<template v-for="backend in item.not_reported_by" :key="`${item.id}-nrb-${backend}`">
|
|
||||||
<NuxtLink :to="'/backend/'+backend" v-text="backend" class="tag"/>
|
|
||||||
|
|
||||||
</template>
|
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
193
src/API/System/Integrity.php
Normal file
193
src/API/System/Integrity.php
Normal file
@@ -0,0 +1,193 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
namespace App\API\System;
|
||||||
|
|
||||||
|
use App\Libs\Attributes\Route\Delete;
|
||||||
|
use App\Libs\Attributes\Route\Get;
|
||||||
|
use App\Libs\Container;
|
||||||
|
use App\Libs\Database\DatabaseInterface as iDB;
|
||||||
|
use App\Libs\Entity\StateInterface as iState;
|
||||||
|
use App\Libs\HTTP_STATUS;
|
||||||
|
use App\Libs\Middlewares\ExceptionHandlerMiddleware;
|
||||||
|
use App\Libs\Traits\APITraits;
|
||||||
|
use DateInterval;
|
||||||
|
use PDO;
|
||||||
|
use Psr\Http\Message\ResponseInterface as iResponse;
|
||||||
|
use Psr\Http\Message\ServerRequestInterface as iRequest;
|
||||||
|
use Psr\SimpleCache\CacheInterface as iCache;
|
||||||
|
use Psr\SimpleCache\InvalidArgumentException;
|
||||||
|
|
||||||
|
final class Integrity
|
||||||
|
{
|
||||||
|
use APITraits;
|
||||||
|
|
||||||
|
public const string URL = '%{api.prefix}/system/integrity';
|
||||||
|
|
||||||
|
private array $dirExists = [];
|
||||||
|
|
||||||
|
private array $checkedFile = [];
|
||||||
|
|
||||||
|
private bool $fromCache = false;
|
||||||
|
private PDO $pdo;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @throws InvalidArgumentException
|
||||||
|
*/
|
||||||
|
public function __construct(private iDB $db, private readonly iCache $cache)
|
||||||
|
{
|
||||||
|
$this->pdo = $this->db->getPDO();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @throws InvalidArgumentException
|
||||||
|
*/
|
||||||
|
#[Get(self::URL . '[/]', middleware: [ExceptionHandlerMiddleware::class], name: 'system.integrity')]
|
||||||
|
public function __invoke(iRequest $request): iResponse
|
||||||
|
{
|
||||||
|
if ($this->cache->has('system.integrity')) {
|
||||||
|
$data = $this->cache->get('system.integrity', []);
|
||||||
|
$this->dirExists = ag($data, 'dir_exists', []);
|
||||||
|
$this->checkedFile = ag($data, 'checked_file', []);
|
||||||
|
$this->fromCache = true;
|
||||||
|
}
|
||||||
|
|
||||||
|
$response = [
|
||||||
|
'items' => [],
|
||||||
|
'fromCache' => $this->fromCache,
|
||||||
|
];
|
||||||
|
|
||||||
|
$sql = "SELECT * FROM state";
|
||||||
|
$stmt = $this->db->getPDO()->prepare($sql);
|
||||||
|
$stmt->execute();
|
||||||
|
|
||||||
|
$base = Container::get(iState::class);
|
||||||
|
|
||||||
|
foreach ($stmt as $row) {
|
||||||
|
$entity = $base::fromArray($row);
|
||||||
|
if (false === $this->checkIntegrity($entity)) {
|
||||||
|
$response['items'][] = $this->formatEntity($entity, true);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
$this->cache->set('system.integrity', [
|
||||||
|
'dir_exists' => $this->dirExists,
|
||||||
|
'checked_file' => $this->checkedFile,
|
||||||
|
], new DateInterval('PT1H'));
|
||||||
|
|
||||||
|
return api_response(HTTP_STATUS::HTTP_OK, $response);
|
||||||
|
}
|
||||||
|
|
||||||
|
private function checkIntegrity(iState $entity): bool
|
||||||
|
{
|
||||||
|
$metadata = $entity->getMetadata();
|
||||||
|
|
||||||
|
if (empty($metadata)) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
$checks = [];
|
||||||
|
|
||||||
|
foreach ($metadata as $backend => $data) {
|
||||||
|
if (!isset($data['path'])) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
$checks[] = [
|
||||||
|
'backend' => $backend,
|
||||||
|
'path' => $data['path'],
|
||||||
|
'status' => true,
|
||||||
|
'message' => '',
|
||||||
|
];
|
||||||
|
}
|
||||||
|
|
||||||
|
if (empty($checks)) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
foreach ($checks as &$check) {
|
||||||
|
$path = $check['path'];
|
||||||
|
$dirName = dirname($path);
|
||||||
|
|
||||||
|
if (false === $this->checkPath($dirName)) {
|
||||||
|
$check['status'] = false;
|
||||||
|
$check['message'] = "File parent directory does not exist.";
|
||||||
|
continue;
|
||||||
|
} else {
|
||||||
|
$check['status'] = true;
|
||||||
|
$check['message'] = "File parent directory exists.";
|
||||||
|
}
|
||||||
|
|
||||||
|
if (false === $this->checkFile($path)) {
|
||||||
|
$check['status'] = false;
|
||||||
|
$check['message'] = "File does not exist.";
|
||||||
|
} else {
|
||||||
|
$check['status'] = true;
|
||||||
|
$check['message'] = "File exists.";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
unset($check);
|
||||||
|
|
||||||
|
foreach ($checks as $check) {
|
||||||
|
if (false === $check['status']) {
|
||||||
|
$entity->setContext('integrity', $checks);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @throws InvalidArgumentException
|
||||||
|
*/
|
||||||
|
#[Delete(self::URL . '[/]', name: 'system.integrity.reset')]
|
||||||
|
public function resetCache(iRequest $request): iResponse
|
||||||
|
{
|
||||||
|
if ($this->cache->has('system.integrity')) {
|
||||||
|
$this->cache->delete('system.integrity');
|
||||||
|
}
|
||||||
|
|
||||||
|
return api_response(HTTP_STATUS::HTTP_OK);
|
||||||
|
}
|
||||||
|
|
||||||
|
private function checkPath(string $dir): bool
|
||||||
|
{
|
||||||
|
$dirs = explode(DIRECTORY_SEPARATOR, $dir);
|
||||||
|
foreach ($dirs as $i => $dir) {
|
||||||
|
$path = implode(DIRECTORY_SEPARATOR, array_slice($dirs, 0, $i + 1));
|
||||||
|
if (empty($path)) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (false === $this->dirExists($path)) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
private function dirExists(string $dir): bool
|
||||||
|
{
|
||||||
|
if (array_key_exists($dir, $this->dirExists)) {
|
||||||
|
return $this->dirExists[$dir];
|
||||||
|
}
|
||||||
|
|
||||||
|
$this->dirExists[$dir] = is_dir($dir);
|
||||||
|
|
||||||
|
return $this->dirExists[$dir];
|
||||||
|
}
|
||||||
|
|
||||||
|
private function checkFile(string $file): bool
|
||||||
|
{
|
||||||
|
if (array_key_exists($file, $this->checkedFile)) {
|
||||||
|
return $this->checkedFile[$file];
|
||||||
|
}
|
||||||
|
|
||||||
|
$this->checkedFile[$file] = file_exists($file);
|
||||||
|
|
||||||
|
return $this->checkedFile[$file];
|
||||||
|
}
|
||||||
|
}
|
||||||
22
src/Libs/Middlewares/ExceptionHandlerMiddleware.php
Normal file
22
src/Libs/Middlewares/ExceptionHandlerMiddleware.php
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
<?php
|
||||||
|
|
||||||
|
declare(strict_types=1);
|
||||||
|
|
||||||
|
namespace App\Libs\Middlewares;
|
||||||
|
|
||||||
|
use Psr\Http\Message\ResponseInterface as iResponse;
|
||||||
|
use Psr\Http\Message\ServerRequestInterface as iRequest;
|
||||||
|
use Psr\Http\Server\MiddlewareInterface as iMiddleware;
|
||||||
|
use Psr\Http\Server\RequestHandlerInterface as iHandler;
|
||||||
|
|
||||||
|
final class ExceptionHandlerMiddleware implements iMiddleware
|
||||||
|
{
|
||||||
|
public function process(iRequest $request, iHandler $handler): iResponse
|
||||||
|
{
|
||||||
|
try {
|
||||||
|
return $handler->handle($request);
|
||||||
|
} catch (\Throwable $e) {
|
||||||
|
return api_error($e->getMessage(), $e->getCode());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user