github/iptv
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Merge pull request #27911 from iptv-org/patch-2025.10.1

Browse Source 
Patch 2025.10.1
This commit is contained in:
Sphinx
2025-10-10 20:21:40 -04:00
committed by GitHub
parent b762d0a233 60fde4cc8c
commit e1659e2b96
214 changed files with 16257 additions and 17871 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
.github/CODE_OF_CONDUCT.md vendored
View File

@@ -1,13 +1,13 @@
# Contributor Code of Conduct # Contributor Code of Conduct
As contributors and maintainers of this project, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities. As contributors and maintainers of this project, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities.
We are committed to making participation in this project a harassment-free experience for everyone, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, or religion. We are committed to making participation in this project a harassment-free experience for everyone, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, or religion.
Examples of unacceptable behavior by participants include the use of sexual language or imagery, derogatory comments or personal attacks, trolling, public or private harassment, insults, or other unprofessional conduct. Examples of unacceptable behavior by participants include the use of sexual language or imagery, derogatory comments or personal attacks, trolling, public or private harassment, insults, or other unprofessional conduct.
Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct. Project maintainers who do not follow the Code of Conduct may be removed from the project team. Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct. Project maintainers who do not follow the Code of Conduct may be removed from the project team.
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by opening an issue or contacting one or more of the project maintainers. Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by opening an issue or contacting one or more of the project maintainers.
This Code of Conduct is adapted from the [Contributor Covenant](http:contributor-covenant.org), version 1.0.0, available at https://www.contributor-covenant.org/version/1/0/0/code-of-conduct.html This Code of Conduct is adapted from the [Contributor Covenant](http:contributor-covenant.org), version 1.0.0, available at https://www.contributor-covenant.org/version/1/0/0/code-of-conduct.html

24
.github/FUNDING.yml vendored
View File

@@ -1,12 +1,12 @@
# These are supported funding model platforms # These are supported funding model platforms
github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2] github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
patreon: # Replace with a single Patreon username patreon: # Replace with a single Patreon username
open_collective: iptv-org open_collective: iptv-org
ko_fi: # Replace with a single Ko-fi username ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: # Replace with a single Liberapay username liberapay: # Replace with a single Liberapay username
issuehunt: # Replace with a single IssueHunt username issuehunt: # Replace with a single IssueHunt username
otechie: # Replace with a single Otechie username otechie: # Replace with a single Otechie username
custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2'] custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']

164
.github/ISSUE_TEMPLATE/1_streams_add.yml vendored
View File

@@ -1,82 +1,82 @@
name: ➕ Add stream name: ➕ Add stream
description: Request to add a new stream link into the playlist description: Request to add a new stream link into the playlist
title: 'Add: ' title: 'Add: '
labels: ['streams:add'] labels: ['streams:add']
body: body:
- type: input - type: input
id: stream_id id: stream_id
attributes: attributes:
label: Stream ID (required) label: Stream ID (required)
description: "ID of the stream consisting of `<channel_id>` or `<channel_id>@<feed_id>`. Full list of supported channels with corresponding ID could be found on [iptv-org.github.io](https://iptv-org.github.io/). If you can't find the channel you want in the list, please let us know through this [form](https://github.com/iptv-org/database/issues/new?assignees=&labels=channels%3Aadd&projects=&template=channels_add.yml&title=Add%3A+) before posting your request." description: "ID of the stream consisting of `<channel_id>` or `<channel_id>@<feed_id>`. Full list of supported channels with corresponding ID could be found on [iptv-org.github.io](https://iptv-org.github.io/). If you can't find the channel you want in the list, please let us know through this [form](https://github.com/iptv-org/database/issues/new?assignees=&labels=channels%3Aadd&projects=&template=channels_add.yml&title=Add%3A+) before posting your request."
placeholder: 'BBCAmerica.us@East' placeholder: 'BBCAmerica.us@East'
validations: validations:
required: true required: true
- type: input - type: input
id: stream_url id: stream_url
attributes: attributes:
label: Stream URL (required) label: Stream URL (required)
description: Link to be added to the playlist description: Link to be added to the playlist
placeholder: 'https://example.com/playlist.m3u8' placeholder: 'https://example.com/playlist.m3u8'
validations: validations:
required: true required: true
- type: dropdown - type: dropdown
id: quality id: quality
attributes: attributes:
label: Quality label: Quality
description: Maximum video resolution available on the link description: Maximum video resolution available on the link
options: options:
- 2160p - 2160p
- 1280p - 1280p
- 1080p - 1080p
- 1080i - 1080i
- 720p - 720p
- 576p - 576p
- 576i - 576i
- 480p - 480p
- 480i - 480i
- 360p - 360p
- type: dropdown - type: dropdown
id: label id: label
attributes: attributes:
label: Label label: Label
description: Is there any reason why the broadcast may not work? description: Is there any reason why the broadcast may not work?
options: options:
- 'Not 24/7' - 'Not 24/7'
- 'Geo-blocked' - 'Geo-blocked'
- type: input - type: input
id: http_user_agent id: http_user_agent
attributes: attributes:
label: HTTP User Agent label: HTTP User Agent
placeholder: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36 Edge/12.246' placeholder: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36 Edge/12.246'
- type: input - type: input
id: http_referrer id: http_referrer
attributes: attributes:
label: HTTP Referrer label: HTTP Referrer
placeholder: 'https://example.com/' placeholder: 'https://example.com/'
- type: textarea - type: textarea
id: directives id: directives
attributes: attributes:
label: Directives label: Directives
description: 'List of directives telling players how to play the stream. Supported `#KODIPROP` and `#VLCOPT`.' description: 'List of directives telling players how to play the stream. Supported `#KODIPROP` and `#EXTVLCOPT`.'
placeholder: '#KODIPROP:inputstream=inputstream.adaptive' placeholder: '#KODIPROP:inputstream=inputstream.adaptive'
- type: textarea - type: textarea
id: notes id: notes
attributes: attributes:
label: Notes label: Notes
description: 'Anything else we should know about this broadcast?' description: 'Anything else we should know about this broadcast?'
- type: checkboxes - type: checkboxes
attributes: attributes:
label: Contributing Guide label: Contributing Guide
description: 'Please read this guide before posting your request' description: 'Please read this guide before posting your request'
options: options:
- label: I have read [Contributing Guide](https://github.com/iptv-org/iptv/blob/master/CONTRIBUTING.md) - label: I have read [Contributing Guide](https://github.com/iptv-org/iptv/blob/master/CONTRIBUTING.md)
required: true required: true

188
.github/ISSUE_TEMPLATE/2_streams_edit.yml vendored
View File

@@ -1,94 +1,94 @@
name: ✏️ Edit stream name: ✏️ Edit stream
description: Request to edit stream description description: Request to edit stream description
title: 'Edit: ' title: 'Edit: '
labels: ['streams:edit'] labels: ['streams:edit']
body: body:
- type: input - type: input
id: stream_url id: stream_url
attributes: attributes:
label: Stream URL (required) label: Stream URL (required)
description: Link to the stream to be updated description: Link to the stream to be updated
placeholder: 'https://lnc-kdfw-fox-aws.tubi.video/index.m3u8' placeholder: 'https://lnc-kdfw-fox-aws.tubi.video/index.m3u8'
validations: validations:
required: true required: true
- type: markdown - type: markdown
attributes: attributes:
value: | value: |
What exactly needs to be changed? To delete an existing value without replacement use the `~` symbol. What exactly needs to be changed? To delete an existing value without replacement use the `~` symbol.
- type: input - type: input
id: new_stream_url id: new_stream_url
attributes: attributes:
label: New Stream URL label: New Stream URL
description: New link to the stream description: New link to the stream
placeholder: 'https://servilive.com:3126/live/tele2000live.m3u8' placeholder: 'https://servilive.com:3126/live/tele2000live.m3u8'
- type: input - type: input
id: stream_id id: stream_id
attributes: attributes:
label: Stream ID label: Stream ID
description: "ID of the stream consisting of `<channel_id>` or `<channel_id>@<feed_id>`. Full list of supported channels with corresponding ID could be found on [iptv-org.github.io](https://iptv-org.github.io/). If you can't find the channel you want in the list, please let us know through this [form](https://github.com/iptv-org/database/issues/new?assignees=&labels=channels%3Aadd&projects=&template=channels_add.yml&title=Add%3A+) before posting your request." description: "ID of the stream consisting of `<channel_id>` or `<channel_id>@<feed_id>`. Full list of supported channels with corresponding ID could be found on [iptv-org.github.io](https://iptv-org.github.io/). If you can't find the channel you want in the list, please let us know through this [form](https://github.com/iptv-org/database/issues/new?assignees=&labels=channels%3Aadd&projects=&template=channels_add.yml&title=Add%3A+) before posting your request."
placeholder: 'BBCAmerica.us@East' placeholder: 'BBCAmerica.us@East'
- type: dropdown - type: dropdown
id: quality id: quality
attributes: attributes:
label: Quality label: Quality
description: Maximum video resolution available on the link description: Maximum video resolution available on the link
options: options:
- 2160p - 2160p
- 1280p - 1280p
- 1080p - 1080p
- 1080i - 1080i
- 720p - 720p
- 576p - 576p
- 576i - 576i
- 480p - 480p
- 480i - 480i
- 360p - 360p
- '~' - '~'
- type: dropdown - type: dropdown
id: label id: label
attributes: attributes:
label: Label label: Label
description: Is there any reason why the broadcast may not work? description: Is there any reason why the broadcast may not work?
options: options:
- 'Not 24/7' - 'Not 24/7'
- 'Geo-blocked' - 'Geo-blocked'
- '~' - '~'
- type: input - type: input
id: http_user_agent id: http_user_agent
attributes: attributes:
label: HTTP User Agent label: HTTP User Agent
placeholder: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36 Edge/12.246' placeholder: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36 Edge/12.246'
- type: input - type: input
id: http_referrer id: http_referrer
attributes: attributes:
label: HTTP Referrer label: HTTP Referrer
placeholder: 'https://example.com/' placeholder: 'https://example.com/'
- type: textarea - type: textarea
id: directives id: directives
attributes: attributes:
label: Directives label: Directives
description: 'List of directives telling players how to play the stream. Supported `#KODIPROP` and `#VLCOPT`.' description: 'List of directives telling players how to play the stream. Supported `#KODIPROP` and `#EXTVLCOPT`.'
placeholder: '#KODIPROP:inputstream=inputstream.adaptive' placeholder: '#KODIPROP:inputstream=inputstream.adaptive'
- type: textarea - type: textarea
id: notes id: notes
attributes: attributes:
label: Notes label: Notes
placeholder: 'Anything else we should know?' placeholder: 'Anything else we should know?'
- type: checkboxes - type: checkboxes
attributes: attributes:
label: Contributing Guide label: Contributing Guide
description: 'Please read this guide before posting your request' description: 'Please read this guide before posting your request'
options: options:
- label: I have read [Contributing Guide](https://github.com/iptv-org/iptv/blob/master/CONTRIBUTING.md) - label: I have read [Contributing Guide](https://github.com/iptv-org/iptv/blob/master/CONTRIBUTING.md)
required: true required: true

98
.github/ISSUE_TEMPLATE/3_streams_report.yml vendored
View File

@@ -1,49 +1,49 @@
name: 🚧 Report a stream name: 🚧 Report a stream
description: Report a broken or unstable stream description: Report a broken or unstable stream
title: 'Report: ' title: 'Report: '
labels: ['streams:remove'] labels: ['streams:remove']
body: body:
- type: markdown - type: markdown
attributes: attributes:
value: | value: |
Please fill out the form as much as you can so we could efficiently process your request. To suggest a new replacement link, use this [form](https://github.com/iptv-org/iptv/issues/new?assignees=&labels=streams:add&projects=&template=1_streams_add.yml&title=Add%3A+). Please fill out the form as much as you can so we could efficiently process your request. To suggest a new replacement link, use this [form](https://github.com/iptv-org/iptv/issues/new?assignees=&labels=streams:add&projects=&template=1_streams_add.yml&title=Add%3A+).
- type: textarea - type: textarea
id: stream_url id: stream_url
attributes: attributes:
label: Stream URL label: Stream URL
description: List all links in question (one per line) description: List all links in question (one per line)
placeholder: 'https://lnc-kdfw-fox-aws.tubi.video/index.m3u8' placeholder: 'https://lnc-kdfw-fox-aws.tubi.video/index.m3u8'
validations: validations:
required: true required: true
- type: dropdown - type: dropdown
id: reason id: reason
attributes: attributes:
label: What happened to the stream? label: What happened to the stream?
options: options:
- Not loading - Not loading
- Constantly interrupts/lagging - Constantly interrupts/lagging
- Stuck at a single frame - Stuck at a single frame
- I see visual artifacts - I see visual artifacts
- Shows looped video - Shows looped video
- No sound - No sound
- Displays a message asking to renew subscription - Displays a message asking to renew subscription
- Other - Other
validations: validations:
required: true required: true
- type: textarea - type: textarea
id: notes id: notes
attributes: attributes:
label: Notes (optional) label: Notes (optional)
placeholder: 'Anything else we should know?' placeholder: 'Anything else we should know?'
- type: checkboxes - type: checkboxes
attributes: attributes:
label: Contributing Guide label: Contributing Guide
description: 'Please read this guide before posting your request' description: 'Please read this guide before posting your request'
options: options:
- label: I have read [Contributing Guide](https://github.com/iptv-org/iptv/blob/master/CONTRIBUTING.md) - label: I have read [Contributing Guide](https://github.com/iptv-org/iptv/blob/master/CONTRIBUTING.md)
required: true required: true

56
.github/ISSUE_TEMPLATE/4_channel-search.yml vendored
View File

@@ -1,28 +1,28 @@
name: 🔍 Channel search name: 🔍 Channel search
description: Ask for help in finding a link to the channel stream. description: Ask for help in finding a link to the channel stream.
title: 'Find: ' title: 'Find: '
labels: ['channel search'] labels: ['channel search']
body: body:
- type: input - type: input
id: stream_id id: stream_id
attributes: attributes:
label: Channel ID (required) label: Stream ID (required)
description: Unique channel ID from [iptv-org.github.io](https://iptv-org.github.io/). If you can't find the channel you want in the list, please let us know through this [form](https://github.com/iptv-org/database/issues/new?assignees=&labels=channels%3Aadd&projects=&template=channels_add.yml&title=Add%3A+) before posting your request. description: Unique ID of the channel and feed from [iptv-org.github.io](https://iptv-org.github.io/). If you cannot find the channel or feed you are looking for in the list, please let us know via one of the [forms](https://github.com/iptv-org/database/issues/new/choose) before posting your request.
placeholder: 'BBCAmericaEast.us' placeholder: 'BBCAmerica.us@East'
validations: validations:
required: true required: true
- type: textarea - type: textarea
id: notes id: notes
attributes: attributes:
label: Notes label: Notes
description: 'Any additional information that may help find a link to the stream faster?' description: 'Any additional information that may help find a link to the stream faster?'
- type: checkboxes - type: checkboxes
attributes: attributes:
label: Contributing Guide label: Contributing Guide
description: 'Please read this guide before posting your request' description: 'Please read this guide before posting your request'
options: options:
- label: I have read [Contributing Guide](https://github.com/iptv-org/iptv/blob/master/CONTRIBUTING.md) - label: I have read [Contributing Guide](https://github.com/iptv-org/iptv/blob/master/CONTRIBUTING.md)
required: true required: true

36
.github/ISSUE_TEMPLATE/5_bug-report.yml vendored
View File

@@ -1,19 +1,19 @@
name: 🐞 Bug report name: 🐞 Bug report
description: Report an error in this repository description: Report an error in this repository
labels: ['bug'] labels: ['bug']
body: body:
- type: textarea - type: textarea
attributes: attributes:
label: Describe your issue label: Describe your issue
description: Please describe the error in as much detail as possible so that we can fix it quickly. description: Please describe the error in as much detail as possible so that we can fix it quickly.
validations: validations:
required: true required: true
- type: checkboxes - type: checkboxes
attributes: attributes:
label: Contributing Guide label: Contributing Guide
description: 'Please read this guide before posting your request' description: 'Please read this guide before posting your request'
options: options:
- label: I have read [Contributing Guide](https://github.com/iptv-org/iptv/blob/master/CONTRIBUTING.md) - label: I have read [Contributing Guide](https://github.com/iptv-org/iptv/blob/master/CONTRIBUTING.md)
required: true required: true

100
.github/ISSUE_TEMPLATE/6_copyright-claim.yml vendored
View File

@@ -1,50 +1,50 @@
name: ©️ Copyright removal request name: ©️ Copyright removal request
description: Request to remove content description: Request to remove content
labels: ['removal request'] labels: ['removal request']
body: body:
- type: input - type: input
attributes: attributes:
label: Your full legal name label: Your full legal name
validations: validations:
required: true required: true
- type: dropdown - type: dropdown
attributes: attributes:
label: Are you the copyright holder or authorized to act on the copyright owner's behalf? label: Are you the copyright holder or authorized to act on the copyright owner's behalf?
description: We cannot process your request unless it is submitted by the copyright owner or an agent authorized to act on behalf of the copyright owner. description: We cannot process your request unless it is submitted by the copyright owner or an agent authorized to act on behalf of the copyright owner.
options: options:
- Yes, I am the copyright holder. - Yes, I am the copyright holder.
- Yes, I am authorized to act on the copyright owner's behalf. - Yes, I am authorized to act on the copyright owner's behalf.
- No. - No.
validations: validations:
required: true required: true
- type: textarea - type: textarea
attributes: attributes:
label: Please describe the nature of your copyright ownership or authorization to act on the owner's behalf. label: Please describe the nature of your copyright ownership or authorization to act on the owner's behalf.
validations: validations:
required: true required: true
- type: textarea - type: textarea
attributes: attributes:
label: Please provide a detailed description of the original copyrighted work that has allegedly been infringed. If possible, include a URL to where it is posted online. label: Please provide a detailed description of the original copyrighted work that has allegedly been infringed. If possible, include a URL to where it is posted online.
validations: validations:
required: true required: true
- type: textarea - type: textarea
attributes: attributes:
label: What content should be removed? Please specify the URL for each item or, if it is an entire file, the file's URL. label: What content should be removed? Please specify the URL for each item or, if it is an entire file, the file's URL.
validations: validations:
required: true required: true
- type: textarea - type: textarea
attributes: attributes:
label: Any additional information we need to know? label: Any additional information we need to know?
- type: checkboxes - type: checkboxes
attributes: attributes:
label: Contributing Guide label: Contributing Guide
description: 'Please read this guide before posting your request' description: 'Please read this guide before posting your request'
options: options:
- label: I have read [Contributing Guide](https://github.com/iptv-org/iptv/blob/master/CONTRIBUTING.md) - label: I have read [Contributing Guide](https://github.com/iptv-org/iptv/blob/master/CONTRIBUTING.md)
required: true required: true

22
.github/ISSUE_TEMPLATE/config.yml vendored
View File

@@ -1,11 +1,11 @@
blank_issues_enabled: false blank_issues_enabled: false
contact_links: contact_links:
- name: 💡 Ideas - name: 💡 Ideas
url: https://github.com/orgs/iptv-org/discussions/categories/ideas url: https://github.com/orgs/iptv-org/discussions/categories/ideas
about: Share ideas for new features about: Share ideas for new features
- name: 🙌 Show and tell - name: 🙌 Show and tell
url: https://github.com/orgs/iptv-org/discussions/categories/show-and-tell url: https://github.com/orgs/iptv-org/discussions/categories/show-and-tell
about: Show off something you've made about: Show off something you've made
- name: ❓ Q&A - name: ❓ Q&A
url: https://github.com/orgs/iptv-org/discussions/categories/q-a url: https://github.com/orgs/iptv-org/discussions/categories/q-a
about: Ask the community for help about: Ask the community for help

77
.github/workflows/check.yml vendored
View File

@@ -1,37 +1,42 @@
name: check name: check
on: on:
workflow_dispatch: workflow_dispatch:
pull_request: pull_request:
types: [opened, synchronize, reopened] types: [opened, synchronize, reopened]
concurrency: concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }} group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true cancel-in-progress: true
jobs: jobs:
check: check:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- name: changed files - name: Get list of changed files
id: files id: files
run: | run: |
git fetch origin master:master git fetch origin master:master
ANY_CHANGED=false ANY_CHANGED=false
ALL_CHANGED_FILES=$(git diff --name-only master -- streams/ | tr '\n' ' ') ALL_CHANGED_FILES=$(git diff --name-only master -- streams/ | tr '\n' ' ')
if [ -n "${ALL_CHANGED_FILES}" ]; then if [ -n "${ALL_CHANGED_FILES}" ]; then
ANY_CHANGED=true ANY_CHANGED=true
fi fi
echo "all_changed_files=$ALL_CHANGED_FILES" >> "$GITHUB_OUTPUT" echo "all_changed_files=$ALL_CHANGED_FILES" >> "$GITHUB_OUTPUT"
echo "any_changed=$ANY_CHANGED" >> "$GITHUB_OUTPUT" echo "any_changed=$ANY_CHANGED" >> "$GITHUB_OUTPUT"
- uses: actions/setup-node@v4 - uses: actions/setup-node@v4
if: ${{ !env.ACT && steps.files.outputs.any_changed == 'true' }} if: steps.files.outputs.any_changed == 'true'
with: with:
node-version: 22 node-version: 22
cache: 'npm' cache: 'npm'
- name: install dependencies - name: Setup .npmrc for GitHub Packages
if: steps.files.outputs.any_changed == 'true' run: |
run: npm install echo "//npm.pkg.github.com/:_authToken=$GITHUB_TOKEN" >> .npmrc
- name: validate echo "@iptv-org:registry=https://npm.pkg.github.com/" >> .npmrc
if: steps.files.outputs.any_changed == 'true' echo "always-auth=true" >> .npmrc
run: | - name: Install dependencies
npm run playlist:lint -- ${{ steps.files.outputs.all_changed_files }} if: steps.files.outputs.any_changed == 'true'
run: npm install
- name: Validate changed files
if: steps.files.outputs.any_changed == 'true'
run: |
npm run playlist:lint -- ${{ steps.files.outputs.all_changed_files }}
npm run playlist:validate -- ${{ steps.files.outputs.all_changed_files }} npm run playlist:validate -- ${{ steps.files.outputs.all_changed_files }}

193
.github/workflows/format.yml vendored
View File

@@ -1,131 +1,62 @@
name: format name: format
on: on:
workflow_dispatch: workflow_dispatch:
# pull_request: # schedule:
# types: [closed] # - cron: "0 12 * * *"
# branches: jobs:
# - master on_trigger:
# schedule: # if: ${{ github.event_name == 'workflow_dispatch' || github.event_name == 'schedule' }}
# - cron: "0 12 * * *" if: ${{ github.event_name == 'workflow_dispatch' }}
jobs: runs-on: ubuntu-latest
on_trigger: steps:
# if: ${{ github.event_name == 'workflow_dispatch' || github.event_name == 'schedule' }} - uses: actions/checkout@v4
if: ${{ github.event_name == 'workflow_dispatch' }} - uses: tibdex/github-app-token@v1.8.2
runs-on: ubuntu-latest if: ${{ !env.ACT }}
steps: id: create-app-token
- uses: actions/checkout@v4 with:
- uses: tibdex/github-app-token@v1.8.2 app_id: ${{ secrets.APP_ID }}
if: ${{ !env.ACT }} private_key: ${{ secrets.APP_PRIVATE_KEY }}
id: create-app-token - uses: actions/checkout@v4
with: if: ${{ !env.ACT }}
app_id: ${{ secrets.APP_ID }} with:
private_key: ${{ secrets.APP_PRIVATE_KEY }} token: ${{ steps.create-app-token.outputs.token }}
- uses: actions/checkout@v4 - uses: actions/setup-node@v4
if: ${{ !env.ACT }} with:
with: node-version: 22
token: ${{ steps.create-app-token.outputs.token }} cache: 'npm'
- uses: actions/setup-node@v4 - name: Setup .npmrc for GitHub Packages
with: run: |
node-version: 22 echo "//npm.pkg.github.com/:_authToken=$GITHUB_TOKEN" >> .npmrc
cache: 'npm' echo "@iptv-org:registry=https://npm.pkg.github.com/" >> .npmrc
- name: setup git echo "always-auth=true" >> .npmrc
run: | - name: Install dependencies
git config user.name "iptv-bot[bot]" run: npm install
git config user.email "84861620+iptv-bot[bot]@users.noreply.github.com" - name: Format internal playlists
- name: install dependencies run: npm run playlist:format
run: npm install - name: Check internal playlists
- name: format internal playlists run: |
run: npm run playlist:format npm run playlist:lint
- name: check internal playlists npm run playlist:validate
run: | - name: Get list of changed files
npm run playlist:lint id: files_after
npm run playlist:validate run: |
- name: changed files ANY_CHANGED=false
id: files_after ALL_CHANGED_FILES=$(git diff --name-only master -- streams/ | tr '\n' ' ')
run: | if [ -n "${ALL_CHANGED_FILES}" ]; then
ANY_CHANGED=false ANY_CHANGED=true
ALL_CHANGED_FILES=$(git diff --name-only master -- streams/ | tr '\n' ' ') fi
if [ -n "${ALL_CHANGED_FILES}" ]; then echo "all_changed_files=$ALL_CHANGED_FILES" >> "$GITHUB_OUTPUT"
ANY_CHANGED=true echo "any_changed=$ANY_CHANGED" >> "$GITHUB_OUTPUT"
fi - name: Setup git
echo "all_changed_files=$ALL_CHANGED_FILES" >> "$GITHUB_OUTPUT" run: |
echo "any_changed=$ANY_CHANGED" >> "$GITHUB_OUTPUT" git config user.name "iptv-bot[bot]"
- name: git status git config user.email "84861620+iptv-bot[bot]@users.noreply.github.com"
run: git status - name: Commit changes to /streams
- name: commit changes if: steps.files_after.outputs.any_changed == 'true'
if: steps.files_after.outputs.any_changed == 'true' run: |
run: | git add streams
git add streams git status
git status git commit -m "[Bot] Format /streams" -m "Committed by [iptv-bot](https://github.com/apps/iptv-bot) via [format](https://github.com/iptv-org/iptv/actions/runs/${{ github.run_id }}) workflow." --no-verify
git commit -m "[Bot] Format /streams" -m "Committed by [iptv-bot](https://github.com/apps/iptv-bot) via [format](https://github.com/iptv-org/iptv/actions/runs/${{ github.run_id }}) workflow." --no-verify - name: Push all changes to the repository
- name: push all changes to the repository if: ${{ !env.ACT && github.ref == 'refs/heads/master' && steps.files_after.outputs.any_changed == 'true' }}
if: ${{ !env.ACT && github.ref == 'refs/heads/master' && steps.files_after.outputs.any_changed == 'true' }} run: git push
run: git push
on_merge:
if: github.event.pull_request.merged == true
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: tibdex/github-app-token@v1.8.2
if: ${{ !env.ACT }}
id: create-app-token
with:
app_id: ${{ secrets.APP_ID }}
private_key: ${{ secrets.APP_PRIVATE_KEY }}
- uses: actions/checkout@v4
if: ${{ !env.ACT }}
with:
token: ${{ steps.create-app-token.outputs.token }}
- name: changed files
id: files
run: |
ANY_CHANGED=false
ALL_CHANGED_FILES=$(git diff --name-only master -- streams/ | tr '\n' ' ')
if [ -n "${ALL_CHANGED_FILES}" ]; then
ANY_CHANGED=true
fi
echo "all_changed_files=$ALL_CHANGED_FILES" >> "$GITHUB_OUTPUT"
echo "any_changed=$ANY_CHANGED" >> "$GITHUB_OUTPUT"
- uses: actions/setup-node@v4
if: ${{ steps.files.outputs.any_changed == 'true' }}
with:
node-version: 22
cache: 'npm'
- name: setup git
if: steps.files.outputs.any_changed == 'true'
run: |
git config user.name "iptv-bot[bot]"
git config user.email "84861620+iptv-bot[bot]@users.noreply.github.com"
- name: install dependencies
if: steps.files.outputs.any_changed == 'true'
run: npm install
- name: format internal playlists
if: steps.files.outputs.any_changed == 'true'
run: npm run playlist:format -- ${{ steps.files.outputs.all_changed_files }}
- name: check internal playlists
if: steps.files.outputs.any_changed == 'true'
run: |
npm run playlist:lint -- ${{ steps.files.outputs.all_changed_files }}
npm run playlist:validate -- ${{ steps.files.outputs.all_changed_files }}
- name: git status
if: steps.files.outputs.any_changed == 'true'
run: git status
- name: changed files
id: files_after
run: |
ANY_CHANGED=false
ALL_CHANGED_FILES=$(git diff --name-only master -- streams/ | tr '\n' ' ')
if [ -n "${ALL_CHANGED_FILES}" ]; then
ANY_CHANGED=true
fi
echo "all_changed_files=$ALL_CHANGED_FILES" >> "$GITHUB_OUTPUT"
echo "any_changed=$ANY_CHANGED" >> "$GITHUB_OUTPUT"
- name: commit changes
if: steps.files_after.outputs.any_changed == 'true'
run: |
git add streams
git status
git commit -m "[Bot] Format /streams" -m "Committed by [iptv-bot](https://github.com/apps/iptv-bot) via [format](https://github.com/iptv-org/iptv/actions/runs/${{ github.run_id }}) workflow." --no-verify
- name: push all changes to the repository
if: ${{ !env.ACT && github.ref == 'refs/heads/master' && steps.files_after.outputs.any_changed == 'true' }}
run: git push

49
.github/workflows/stale.yml vendored
View File

@@ -1,25 +1,26 @@
name: stale name: stale
on: on:
workflow_dispatch: workflow_dispatch:
schedule: schedule:
- cron: '0 0 * * *' - cron: '0 0 * * *'
permissions: permissions:
issues: write actions: write
jobs: issues: write
stale: jobs:
runs-on: ubuntu-latest stale:
steps: runs-on: ubuntu-latest
- uses: tibdex/github-app-token@v1.8.2 steps:
id: create-app-token - uses: tibdex/github-app-token@v1.8.2
with: id: create-app-token
app_id: ${{ secrets.APP_ID }} with:
private_key: ${{ secrets.APP_PRIVATE_KEY }} app_id: ${{ secrets.APP_ID }}
- uses: actions/stale@v9 private_key: ${{ secrets.APP_PRIVATE_KEY }}
with: - uses: actions/stale@v9
repo-token: ${{ steps.create-app-token.outputs.token }} with:
days-before-stale: 180 repo-token: ${{ steps.create-app-token.outputs.token }}
days-before-close: 7 days-before-stale: 180
operations-per-run: 500 days-before-close: 7
stale-issue-label: 'stale' operations-per-run: 500
any-of-issue-labels: 'channel search' stale-issue-label: 'stale'
any-of-issue-labels: 'channel search'
close-issue-message: 'This request has been closed because it has been inactive for more than 180 days.' close-issue-message: 'This request has been closed because it has been inactive for more than 180 days.'

167
.github/workflows/update.yml vendored
View File

@@ -1,82 +1,85 @@
name: update name: update
on: on:
workflow_dispatch: workflow_dispatch:
schedule: schedule:
- cron: '0 0 * * *' - cron: '0 0 * * *'
jobs: jobs:
main: main:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v3 - uses: actions/checkout@v4
- uses: tibdex/github-app-token@v1.8.2 - uses: tibdex/github-app-token@v1.8.2
if: ${{ !env.ACT }} if: ${{ !env.ACT }}
id: create-app-token id: create-app-token
with: with:
app_id: ${{ secrets.APP_ID }} app_id: ${{ secrets.APP_ID }}
private_key: ${{ secrets.APP_PRIVATE_KEY }} private_key: ${{ secrets.APP_PRIVATE_KEY }}
- uses: actions/checkout@v3 - uses: actions/checkout@v4
if: ${{ !env.ACT }} if: ${{ !env.ACT }}
with: with:
token: ${{ steps.create-app-token.outputs.token }} token: ${{ steps.create-app-token.outputs.token }}
- name: setup git - uses: actions/setup-node@v4
run: | with:
git config user.name "iptv-bot[bot]" node-version: 22
git config user.email "84861620+iptv-bot[bot]@users.noreply.github.com" cache: 'npm'
- uses: actions/setup-node@v3 - name: Setup .npmrc for GitHub Packages
if: ${{ !env.ACT }} run: |
with: echo "//npm.pkg.github.com/:_authToken=$GITHUB_TOKEN" >> .npmrc
node-version: 20 echo "@iptv-org:registry=https://npm.pkg.github.com/" >> .npmrc
cache: 'npm' echo "always-auth=true" >> .npmrc
- name: install dependencies - name: Install dependencies
run: npm install run: npm install
- name: update internal playlists - name: Update internal playlists
run: npm run playlist:update --silent >> $GITHUB_OUTPUT run: npm run playlist:update --silent >> $GITHUB_OUTPUT
id: playlist-update id: playlist-update
- name: check internal playlists - name: Check internal playlists
run: | run: |
npm run playlist:lint npm run playlist:lint
npm run playlist:validate npm run playlist:validate
- name: generate public playlists - name: Generate public playlists
run: npm run playlist:generate run: npm run playlist:generate
- name: generate .api/streams.json - name: Generate .api/streams.json
run: npm run api:generate run: npm run api:generate
- name: update readme - name: Update readme
run: npm run readme:update run: npm run readme:update
- run: git status - name: Setup git
- name: commit changes to /streams run: |
run: | git config user.name "iptv-bot[bot]"
git add streams git config user.email "84861620+iptv-bot[bot]@users.noreply.github.com"
git status - name: Commit changes to /streams
git commit --allow-empty -m "[Bot] Update /streams" -m "Committed by [iptv-bot](https://github.com/apps/iptv-bot) via [update](https://github.com/iptv-org/iptv/actions/runs/${{ github.run_id }}) workflow." -m "${{ steps.playlist-update.outputs.OUTPUT }}" --no-verify run: |
- name: commit changes to playlists.md git add streams
run: | git status
git add PLAYLISTS.md git commit --allow-empty -m "[Bot] Update /streams" -m "Committed by [iptv-bot](https://github.com/apps/iptv-bot) via [update](https://github.com/iptv-org/iptv/actions/runs/${{ github.run_id }}) workflow." -m "${{ steps.playlist-update.outputs.OUTPUT }}" --no-verify
git status - name: Commit changes to PLAYLIST.md
git commit --allow-empty -m "[Bot] Update PLAYLISTS.md" -m "Committed by [iptv-bot](https://github.com/apps/iptv-bot) via [update](https://github.com/iptv-org/iptv/actions/runs/${{ github.run_id }}) workflow." --no-verify run: |
- name: push all changes to the repository git add PLAYLISTS.md
if: ${{ !env.ACT && github.ref == 'refs/heads/master' }} git status
run: git push git commit --allow-empty -m "[Bot] Update PLAYLISTS.md" -m "Committed by [iptv-bot](https://github.com/apps/iptv-bot) via [update](https://github.com/iptv-org/iptv/actions/runs/${{ github.run_id }}) workflow." --no-verify
- name: deploy public playlists to github pages - name: Push all changes to the repository
uses: JamesIves/github-pages-deploy-action@4.1.1 if: ${{ !env.ACT && github.ref == 'refs/heads/master' }}
if: ${{ !env.ACT && github.ref == 'refs/heads/master' }} run: git push
with: - name: Deploy public playlists to GitHub Pages
repository-name: iptv-org/iptv uses: JamesIves/github-pages-deploy-action@4.1.1
branch: gh-pages if: ${{ !env.ACT && github.ref == 'refs/heads/master' }}
folder: .gh-pages with:
token: ${{ steps.create-app-token.outputs.token }} repository-name: iptv-org/iptv
git-config-name: iptv-bot[bot] branch: gh-pages
git-config-email: 84861620+iptv-bot[bot]@users.noreply.github.com folder: .gh-pages
commit-message: '[Bot] Deploy to GitHub Pages' token: ${{ steps.create-app-token.outputs.token }}
clean: true git-config-name: iptv-bot[bot]
- name: move .api/streams.json to iptv-org/api git-config-email: 84861620+iptv-bot[bot]@users.noreply.github.com
uses: JamesIves/github-pages-deploy-action@4.1.1 commit-message: '[Bot] Deploy to GitHub Pages'
if: ${{ !env.ACT && github.ref == 'refs/heads/master' }} clean: true
with: - name: Move .api/streams.json to iptv-org/api
repository-name: iptv-org/api uses: JamesIves/github-pages-deploy-action@4.1.1
branch: gh-pages if: ${{ !env.ACT && github.ref == 'refs/heads/master' }}
folder: .api with:
token: ${{ steps.create-app-token.outputs.token }} repository-name: iptv-org/api
git-config-name: iptv-bot[bot] branch: gh-pages
git-config-email: 84861620+iptv-bot[bot]@users.noreply.github.com folder: .api
commit-message: '[Bot] Deploy to iptv-org/api' token: ${{ steps.create-app-token.outputs.token }}
clean: false git-config-name: iptv-bot[bot]
git-config-email: 84861620+iptv-bot[bot]@users.noreply.github.com
commit-message: '[Bot] Deploy to iptv-org/api'
clean: false

16
.gitignore vendored
View File

@@ -1,9 +1,9 @@
node_modules node_modules
.artifacts .artifacts
.secrets .secrets
.actrc .actrc
.DS_Store .DS_Store
/.gh-pages/ /.gh-pages/
/.api/ /.api/
.env .env
/temp/ /temp/

6
.readme/.gitignore vendored
View File

@@ -1,4 +1,4 @@
_categories.md _categories.md
_countries.md _countries.md
_languages.md _languages.md
_regions.md _regions.md

176
.readme/template.md
View File

@@ -1,88 +1,88 @@
## Playlists ## Playlists
There are several versions of playlists that differ in the way they are grouped. As of January 30th, 2024, we have stopped distributing NSFW channels. For more information, please look at [this issue](https://github.com/iptv-org/iptv/issues/15723). There are several versions of playlists that differ in the way they are grouped. As of January 30th, 2024, we have stopped distributing NSFW channels. For more information, please look at [this issue](https://github.com/iptv-org/iptv/issues/15723).
### Grouped by category ### Grouped by category
Playlists in which channels are grouped by category. Playlists in which channels are grouped by category.
<details> <details>
<summary>Expand</summary> <summary>Expand</summary>
<br> <br>
``` ```
https://iptv-org.github.io/iptv/index.category.m3u https://iptv-org.github.io/iptv/index.category.m3u
``` ```
Same thing, but split up into separate files: Same thing, but split up into separate files:
<!-- prettier-ignore --> <!-- prettier-ignore -->
#include "./.readme/_categories.md" #include "./.readme/_categories.md"
</details> </details>
### Grouped by language ### Grouped by language
Playlists in which channels are grouped by the language in which they are broadcast. Playlists in which channels are grouped by the language in which they are broadcast.
<details> <details>
<summary>Expand</summary> <summary>Expand</summary>
<br> <br>
``` ```
https://iptv-org.github.io/iptv/index.language.m3u https://iptv-org.github.io/iptv/index.language.m3u
``` ```
Same thing, but split up into separate files: Same thing, but split up into separate files:
<!-- prettier-ignore --> <!-- prettier-ignore -->
#include "./.readme/_languages.md" #include "./.readme/_languages.md"
</details> </details>
### Grouped by broadcast area ### Grouped by broadcast area
Playlists in which channels are grouped by broadcast area. Playlists in which channels are grouped by broadcast area.
<details> <details>
<summary>Expand</summary> <summary>Expand</summary>
#### Countries #### Countries
``` ```
https://iptv-org.github.io/iptv/index.country.m3u https://iptv-org.github.io/iptv/index.country.m3u
``` ```
Same thing, but split up into separate files: Same thing, but split up into separate files:
<!-- prettier-ignore --> <!-- prettier-ignore -->
#include "./.readme/_countries.md" #include "./.readme/_countries.md"
#### Regions #### Regions
<!-- prettier-ignore --> <!-- prettier-ignore -->
#include "./.readme/_regions.md" #include "./.readme/_regions.md"
</details> </details>
### Grouped by sources ### Grouped by sources
Playlists in which channels are grouped by broadcast source. Playlists in which channels are grouped by broadcast source.
<details> <details>
<summary>Expand</summary> <summary>Expand</summary>
<br> <br>
To use the playlist, simply replace `<FILENAME>` in the link below with the name of one of the files in the [streams](streams) folder. To use the playlist, simply replace `<FILENAME>` in the link below with the name of one of the files in the [streams](streams) folder.
``` ```
https://iptv-org.github.io/iptv/sources/<FILENAME>.m3u https://iptv-org.github.io/iptv/sources/<FILENAME>.m3u
``` ```
</details> </details>
Also, any of our internal playlists are available in raw form (without any filtering or sorting) at this link: Also, any of our internal playlists are available in raw form (without any filtering or sorting) at this link:
``` ```
https://iptv-org.github.io/iptv/raw/<FILENAME>.m3u https://iptv-org.github.io/iptv/raw/<FILENAME>.m3u
``` ```

430
CONTRIBUTING.md
View File

@@ -1,215 +1,215 @@
# Contributing Guide # Contributing Guide
- [How to?](#how-to) - [How to?](#how-to)
- [Stream Description Scheme](#stream-description-scheme) - [Stream Description Scheme](#stream-description-scheme)
- [Project Structure](#project-structure) - [Project Structure](#project-structure)
- [Scripts](#scripts) - [Scripts](#scripts)
- [Workflows](#workflows) - [Workflows](#workflows)
## How to? ## How to?
### How to add a new stream link to a playlists? ### How to add a new stream link to a playlists?
You have several options: You have several options:
1. Create a new request using this [form](https://github.com/iptv-org/iptv/issues/new?assignees=&labels=streams:add&projects=&template=1_streams_add.yml&title=Add%3A+) and if approved, the link will automatically be added to the playlist on the next update. 1. Create a new request using this [form](https://github.com/iptv-org/iptv/issues/new?assignees=&labels=streams:add&projects=&template=1_streams_add.yml&title=Add%3A+) and if approved, the link will automatically be added to the playlist on the next update.
2. Add the link to the playlist directly using a [pull request](https://github.com/iptv-org/iptv/pulls). 2. Add the link to the playlist directly using a [pull request](https://github.com/iptv-org/iptv/pulls).
Regardless of which option you choose, before posting your request please do the following: Regardless of which option you choose, before posting your request please do the following:
- Make sure the link you want to add works stably. To check this, open it in one of the players (for example, [VLC player](https://www.videolan.org/vlc/index.html)) and watch the broadcast for at least a minute (some test streams are interrupted after 15-30 seconds). - Make sure the link you want to add works stably. To check this, open it in one of the players (for example, [VLC player](https://www.videolan.org/vlc/index.html)) and watch the broadcast for at least a minute (some test streams are interrupted after 15-30 seconds).
- Make sure the link is not already in the playlist. This can be done by [searching](https://github.com/search?q=repo%3Aiptv-org%2Fiptv+http%3A%2F%2Fexample.com&type=code) the repository. - Make sure the link is not already in the playlist. This can be done by [searching](https://github.com/search?q=repo%3Aiptv-org%2Fiptv+http%3A%2F%2Fexample.com&type=code) the repository.
- Find the ID of the channel you want on [iptv-org.github.io](https://iptv-org.github.io/). If your desired channel is not on the list you can leave a request to add it [here](https://github.com/iptv-org/database/issues/new/choose). - Find the ID of the channel you want on [iptv-org.github.io](https://iptv-org.github.io/). If your desired channel is not on the list you can leave a request to add it [here](https://github.com/iptv-org/database/issues/new/choose).
- Make sure the channel is not blocklisted. It can also be done through [iptv-org.github.io](https://iptv-org.github.io/). - Make sure the channel is not blocklisted. It can also be done through [iptv-org.github.io](https://iptv-org.github.io/).
- The link does not lead to the Xtream Codes server. [Why don't you accept links to Xtream Codes server?](FAQ.md#why-dont-you-accept-links-to-xtream-codes-server) - The link does not lead to the Xtream Codes server. [Why don't you accept links to Xtream Codes server?](FAQ.md#why-dont-you-accept-links-to-xtream-codes-server)
- If you know that the broadcast only works in certain countries or it is periodically interrupted, do not forget to indicate this in the request. - If you know that the broadcast only works in certain countries or it is periodically interrupted, do not forget to indicate this in the request.
A requests without a valid stream ID or working link to the stream will be closed immediately. A requests without a valid stream ID or working link to the stream will be closed immediately.
Note all links in playlists are sorted automatically by scripts so there is no need to sort them manually. For more info, see [Scripts](#scripts). Note all links in playlists are sorted automatically by scripts so there is no need to sort them manually. For more info, see [Scripts](#scripts).
### How to fix the stream description? ### How to fix the stream description?
Most of the stream description (channel name, feed name, categories, languages, broadcast area, logo) we load from the [iptv-org/database](https://github.com/iptv-org/database) using the stream ID. Most of the stream description (channel name, feed name, categories, languages, broadcast area, logo) we load from the [iptv-org/database](https://github.com/iptv-org/database) using the stream ID.
So first of all, make sure that the desired stream has the correct ID. A full list of all supported channels and their corresponding IDs can be found on [iptv-org.github.io](https://iptv-org.github.io/). To change the stream ID of any link in the playlist, just fill out this [form](https://github.com/iptv-org/iptv/issues/new?assignees=&labels=streams%3Aedit&projects=&template=2_streams_edit.yml&title=Edit%3A+). So first of all, make sure that the desired stream has the correct ID. A full list of all supported channels and their corresponding IDs can be found on [iptv-org.github.io](https://iptv-org.github.io/). To change the stream ID of any link in the playlist, just fill out this [form](https://github.com/iptv-org/iptv/issues/new?assignees=&labels=streams%3Aedit&projects=&template=2_streams_edit.yml&title=Edit%3A+).
If, however, you have found an error in the database itself, this is the place to go: [How to edit channel description?](https://github.com/iptv-org/database/blob/master/CONTRIBUTING.md#how-to-edit-channel-description) If, however, you have found an error in the database itself, this is the place to go: [How to edit channel description?](https://github.com/iptv-org/database/blob/master/CONTRIBUTING.md#how-to-edit-channel-description)
### How to distinguish a link to an Xtream Codes server from a regular one? ### How to distinguish a link to an Xtream Codes server from a regular one?
Most of them have this form: Most of them have this form:
`http(s)://{hostname}:{port}/{username}/{password}/{channelID}` (port is often `25461`) `http(s)://{hostname}:{port}/{username}/{password}/{channelID}` (port is often `25461`)
To make sure that the link leads to the Xtream Codes server, copy the `hostname`, `port`, `username` and `password` into the link below and try to open it in a browser: To make sure that the link leads to the Xtream Codes server, copy the `hostname`, `port`, `username` and `password` into the link below and try to open it in a browser:
`http(s)://{hostname}:{port}/panel_api.php?username={username}&password={password}` `http(s)://{hostname}:{port}/panel_api.php?username={username}&password={password}`
If the link answers, you're with an Xtream Codes server. If the link answers, you're with an Xtream Codes server.
### How to report a broken stream? ### How to report a broken stream?
Fill out this [form](https://github.com/iptv-org/iptv/issues/new?assignees=&labels=streams:remove&projects=&template=3_streams_report.yml&title=Broken%3A+) and as soon as a working replacement appears, we will add it to the playlist or at least remove the non-working one. Fill out this [form](https://github.com/iptv-org/iptv/issues/new?assignees=&labels=streams:remove&projects=&template=3_streams_report.yml&title=Broken%3A+) and as soon as a working replacement appears, we will add it to the playlist or at least remove the non-working one.
The only thing before publishing your report is to make sure that: The only thing before publishing your report is to make sure that:
- The link is still in our playlists. You can verify this by [searching](https://github.com/search?q=repo%3Aiptv-org%2Fiptv+http%3A%2F%2Fexample.com&type=code) the repository. - The link is still in our playlists. You can verify this by [searching](https://github.com/search?q=repo%3Aiptv-org%2Fiptv+http%3A%2F%2Fexample.com&type=code) the repository.
- The link really doesn't work and is not just [geo-blocked](https://en.wikipedia.org/wiki/Geo-blocking). To check this, you can either use a [VPN](https://en.wikipedia.org/wiki/Virtual_private_network) or services such as [streamtest.in](https://streamtest.in/). - The link really doesn't work and is not just [geo-blocked](https://en.wikipedia.org/wiki/Geo-blocking). To check this, you can either use a [VPN](https://en.wikipedia.org/wiki/Virtual_private_network) or services such as [streamtest.in](https://streamtest.in/).
An issue without a valid link will be closed immediately. An issue without a valid link will be closed immediately.
### How to find a broken stream? ### How to find a broken stream?
For starters, you can just try to open the playlist in [VLC player](https://www.videolan.org/vlc/). The player outputs all errors to the log (Tools -> Messages) so you'll be able to determine pretty accurately why a link isn't working. For starters, you can just try to open the playlist in [VLC player](https://www.videolan.org/vlc/). The player outputs all errors to the log (Tools -> Messages) so you'll be able to determine pretty accurately why a link isn't working.
Another way to test links is to use the NPM script. To do this, first make sure you have [Node.js](https://nodejs.org/en) installed on your system. Then go to the `iptv` folder using [Console](https://en.wikipedia.org/wiki/Windows_Console) (or [Terminal](<https://en.wikipedia.org/wiki/Terminal_(macOS)>) if you have macOS) and run the command: Another way to test links is to use the NPM script. To do this, first make sure you have [Node.js](https://nodejs.org/en) installed on your system. Then go to the `iptv` folder using [Console](https://en.wikipedia.org/wiki/Windows_Console) (or [Terminal](<https://en.wikipedia.org/wiki/Terminal_(macOS)>) if you have macOS) and run the command:
```sh ```sh
npm run playlist:test path/to/playlist.m3u npm run playlist:test path/to/playlist.m3u
``` ```
This command will run an automatic check of all links in the playlist and display their status: This command will run an automatic check of all links in the playlist and display their status:
```sh ```sh
npm run playlist:test streams/fr.m3u npm run playlist:test streams/fr.m3u
streams/fr.m3u streams/fr.m3u
┌─────┬───────────────────────────┬──────────────────────────────────────────────────────────────────────────────────────────────────────┬───────────────────────────┐ ┌─────┬───────────────────────────┬──────────────────────────────────────────────────────────────────────────────────────────────────────┬───────────────────────────┐
│ │ tvg-id │ url │ status │ │ │ tvg-id │ url │ status │
├─────┼───────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────┤ ├─────┼───────────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────┤
│ 0 │ 6ter.fr │ https://origin-caf900c010ea8046.live.6cloud.fr/out/v1/29c7a579af3348b48230f76cd75699a5/dash_short... │ LOADING... │ │ 0 │ 6ter.fr │ https://origin-caf900c010ea8046.live.6cloud.fr/out/v1/29c7a579af3348b48230f76cd75699a5/dash_short... │ LOADING... │
│ 1 │ 20MinutesTV.fr │ https://lives.digiteka.com/stream/86d3e867-a272-496b-8412-f59aa0104771/index.m3u8 │ FFMPEG_STREAMS_NOT_FOUND │ │ 1 │ 20MinutesTV.fr │ https://lives.digiteka.com/stream/86d3e867-a272-496b-8412-f59aa0104771/index.m3u8 │ FFMPEG_STREAMS_NOT_FOUND │
│ 2 │ │ https://video1.getstreamhosting.com:1936/8420/8420/playlist.m3u8 │ OK │ │ 2 │ │ https://video1.getstreamhosting.com:1936/8420/8420/playlist.m3u8 │ OK │
│ 3 │ ADNTVPlus.fr │ https://samsunguk-adn-samsung-fre-qfrlc.amagi.tv/playlist/samsunguk-adn-samsung-fre/playlist.m3u8 │ HTTP_FORBIDDEN │ │ 3 │ ADNTVPlus.fr │ https://samsunguk-adn-samsung-fre-qfrlc.amagi.tv/playlist/samsunguk-adn-samsung-fre/playlist.m3u8 │ HTTP_FORBIDDEN │
│ 4 │ Africa24.fr │ https://edge12.vedge.infomaniak.com/livecast/ik:africa24/manifest.m3u8 │ OK │ │ 4 │ Africa24.fr │ https://edge12.vedge.infomaniak.com/livecast/ik:africa24/manifest.m3u8 │ OK │
│ 5 │ Africa24English.fr │ https://edge17.vedge.infomaniak.com/livecast/ik:africa24sport/manifest.m3u8 │ OK │ │ 5 │ Africa24English.fr │ https://edge17.vedge.infomaniak.com/livecast/ik:africa24sport/manifest.m3u8 │ OK │
│ 6 │ AfricanewsEnglish.fr │ https://37c774660687468c821a51190046facf.mediatailor.us-east-1.amazonaws.com/v1/master/04fd913bb2... │ HTTP_GATEWAY_TIMEOUT │ │ 6 │ AfricanewsEnglish.fr │ https://37c774660687468c821a51190046facf.mediatailor.us-east-1.amazonaws.com/v1/master/04fd913bb2... │ HTTP_GATEWAY_TIMEOUT │
│ 7 │ AlpedHuezTV.fr │ https://edge.vedge.infomaniak.com/livecast/ik:adhtv/chunklist.m3u8 │ HTTP_NOT_FOUND │ │ 7 │ AlpedHuezTV.fr │ https://edge.vedge.infomaniak.com/livecast/ik:adhtv/chunklist.m3u8 │ HTTP_NOT_FOUND │
``` ```
After that, all you have to do is report any broken streams you find. After that, all you have to do is report any broken streams you find.
### How to replace a broken stream? ### How to replace a broken stream?
This can be done either by filling out this [form](https://github.com/iptv-org/iptv/issues/new?assignees=&labels=streams%3Aedit&projects=&template=2_streams_edit.yml&title=Edit%3A+). This can be done either by filling out this [form](https://github.com/iptv-org/iptv/issues/new?assignees=&labels=streams%3Aedit&projects=&template=2_streams_edit.yml&title=Edit%3A+).
Either by directly updating the files in the [/streams](/streams) folder and then creating a [pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests). Either by directly updating the files in the [/streams](/streams) folder and then creating a [pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests).
### How to remove my channel from playlist? ### How to remove my channel from playlist?
To request removal of a link to a channel from the repository, you need to fill out this [form](https://github.com/iptv-org/iptv/issues/new?assignees=&labels=removal+request&projects=&template=6_copyright-claim.yml&title=Remove%3A+) and wait for the request to be reviewed (this usually takes no more than 1 business day). And if the request is approved, links to the channel will be immediately removed from the repository. To request removal of a link to a channel from the repository, you need to fill out this [form](https://github.com/iptv-org/iptv/issues/new?assignees=&labels=removal+request&projects=&template=6_copyright-claim.yml&title=Remove%3A+) and wait for the request to be reviewed (this usually takes no more than 1 business day). And if the request is approved, links to the channel will be immediately removed from the repository.
The channel will also be added to our [blocklist](https://github.com/iptv-org/database/blob/master/data/blocklist.csv) to avoid its appearance in our playlists in the future. The channel will also be added to our [blocklist](https://github.com/iptv-org/database/blob/master/data/blocklist.csv) to avoid its appearance in our playlists in the future.
Please note that we only accept removal requests from channel owners and their official representatives, all other requests will be closed immediately. Please note that we only accept removal requests from channel owners and their official representatives, all other requests will be closed immediately.
## Stream Description Scheme ## Stream Description Scheme
For a stream to be approved, its description must follow this template: For a stream to be approved, its description must follow this template:
``` ```
#EXTINF:-1 tvg-id="STREAM_ID",STREAM_TITLE (QUALITY) [LABEL] #EXTINF:-1 tvg-id="STREAM_ID",STREAM_TITLE (QUALITY) [LABEL]
STREAM_URL STREAM_URL
``` ```
| Attribute | Description | Required | Valid values | | Attribute | Description | Required | Valid values |
| -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | -------------------------------------------- | | -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | -------------------------------------------- |
| `STREAM_ID` | Stream ID consisting of channel ID and feed ID. Full list of supported channels with corresponding ID could be found on [iptv-org.github.io](https://iptv-org.github.io/). | Optional | `<channel_id>` or `<channel_id>@<feed_id>` | | `STREAM_ID` | Stream ID consisting of channel ID and feed ID. Full list of supported channels with corresponding ID could be found on [iptv-org.github.io](https://iptv-org.github.io/). | Optional | `<channel_id>` or `<channel_id>@<feed_id>` |
| `STREAM_TITLE` | Stream title consisting of channel name and feed name. May contain any characters except: `,`, `[`, `]`. | Required | - | | `STREAM_TITLE` | Stream title consisting of channel name and feed name. May contain any characters except: `,`, `[`, `]`. | Required | - |
| `QUALITY` | Maximum stream quality. | Optional | `2160p`, `1080p`, `720p`, `480p`, `360p` etc | | `QUALITY` | Maximum stream quality. | Optional | `2160p`, `1080p`, `720p`, `480p`, `360p` etc |
| `LABEL` | Specified in cases where the broadcast for some reason may not be available to some users. | Optional | `Geo-blocked` or `Not 24/7` | | `LABEL` | Specified in cases where the broadcast for some reason may not be available to some users. | Optional | `Geo-blocked` or `Not 24/7` |
| `STREAM_URL` | Stream URL. | Required | - | | `STREAM_URL` | Stream URL. | Required | - |
Example: Example:
```xml ```xml
#EXTINF:-1 tvg-id="ExampleTV.us@East",Example TV East (720p) [Not 24/7] #EXTINF:-1 tvg-id="ExampleTV.us@East",Example TV East (720p) [Not 24/7]
https://example.com/playlist.m3u8 https://example.com/playlist.m3u8
``` ```
Also, if necessary, you can specify custom [HTTP User-Agent](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent) and [HTTP Referrer](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referer) through additional attributes: Also, if necessary, you can specify custom [HTTP User-Agent](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent) and [HTTP Referrer](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referer) through additional attributes:
```xml ```xml
#EXTINF:-1 tvg-id="ExampleTV.us" http-referrer="http://example.com/" http-user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64)",Example TV #EXTINF:-1 tvg-id="ExampleTV.us" http-referrer="http://example.com/" http-user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64)",Example TV
http://example.com/stream.m3u8 http://example.com/stream.m3u8
``` ```
or use player-specific directives: or use player-specific directives:
_VLC_ _VLC_
```xml ```xml
#EXTINF:-1 tvg-id="ExampleTV.us@VLC",Example TV #EXTINF:-1 tvg-id="ExampleTV.us@VLC",Example TV
#EXTVLCOPT:http-referrer=http://example.com/ #EXTVLCOPT:http-referrer=http://example.com/
#EXTVLCOPT:http-user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) #EXTVLCOPT:http-user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64)
http://example.com/stream.m3u8 http://example.com/stream.m3u8
``` ```
_Kodi_ _Kodi_
```xml ```xml
#EXTINF:-1 tvg-id="ExampleTV.us@Kodi",Example TV #EXTINF:-1 tvg-id="ExampleTV.us@Kodi",Example TV
#KODIPROP:inputstream=inputstream.adaptive #KODIPROP:inputstream=inputstream.adaptive
#KODIPROP:inputstream.adaptive.stream_headers=Referer=http://example.com/&amp;User-Agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) #KODIPROP:inputstream.adaptive.stream_headers=Referer=http://example.com/&amp;User-Agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64)
http://example.com/stream.m3u8 http://example.com/stream.m3u8
``` ```
## Project Structure ## Project Structure
- `.github/` - `.github/`
- `ISSUE_TEMPLATE/`: issue templates for the repository. - `ISSUE_TEMPLATE/`: issue templates for the repository.
- `workflows`: contains [GitHub actions](https://docs.github.com/en/actions/quickstart) workflows. - `workflows`: contains [GitHub actions](https://docs.github.com/en/actions/quickstart) workflows.
- `CODE_OF_CONDUCT.md`: rules you shouldn't break if you don't want to get banned. - `CODE_OF_CONDUCT.md`: rules you shouldn't break if you don't want to get banned.
- `.readme/` - `.readme/`
- `config.json`: config for the `markdown-include` package, which is used to compile everything into one `PLAYLISTS.md` file. - `config.json`: config for the `markdown-include` package, which is used to compile everything into one `PLAYLISTS.md` file.
- `preview.png`: image displayed in the `README.md`. - `preview.png`: image displayed in the `README.md`.
- `template.md`: template for `PLAYLISTS.md`. - `template.md`: template for `PLAYLISTS.md`.
- `scripts/`: contains all scripts used in the repository. - `scripts/`: contains all scripts used in the repository.
- `streams/`: contains all streams broken down by the country from which they are broadcast. - `streams/`: contains all streams broken down by the country from which they are broadcast.
- `tests/`: contains tests to check the scripts. - `tests/`: contains tests to check the scripts.
- `CONTRIBUTING.md`: file you are currently reading. - `CONTRIBUTING.md`: file you are currently reading.
- `PLAYLISTS.md`: auto-updated list of available playlists. - `PLAYLISTS.md`: auto-updated list of available playlists.
- `README.md`: project description. - `README.md`: project description.
## Scripts ## Scripts
These scripts are created to automate routine processes in the repository and make it a bit easier to maintain. These scripts are created to automate routine processes in the repository and make it a bit easier to maintain.
For scripts to work, you must have [Node.js](https://nodejs.org/en) installed on your computer. For scripts to work, you must have [Node.js](https://nodejs.org/en) installed on your computer.
To run scripts use the `npm run <script-name>` command. To run scripts use the `npm run <script-name>` command.
- `act:check`: allows to run the [check](https://github.com/iptv-org/iptv/blob/master/.github/workflows/check.yml) workflow locally. Depends on [nektos/act](https://github.com/nektos/act). - `act:check`: allows to run the [check](https://github.com/iptv-org/iptv/blob/master/.github/workflows/check.yml) workflow locally. Depends on [nektos/act](https://github.com/nektos/act).
- `act:format`: allows to test the [format](https://github.com/iptv-org/iptv/blob/master/.github/workflows/update.yml) workflow locally. Depends on [nektos/act](https://github.com/nektos/act). - `act:format`: allows to test the [format](https://github.com/iptv-org/iptv/blob/master/.github/workflows/update.yml) workflow locally. Depends on [nektos/act](https://github.com/nektos/act).
- `act:update`: allows to test the [update](https://github.com/iptv-org/iptv/blob/master/.github/workflows/update.yml) workflow locally. Depends on [nektos/act](https://github.com/nektos/act). - `act:update`: allows to test the [update](https://github.com/iptv-org/iptv/blob/master/.github/workflows/update.yml) workflow locally. Depends on [nektos/act](https://github.com/nektos/act).
- `api:load`: downloads the latest channel and stream data from the [iptv-org/api](https://github.com/iptv-org/api). - `api:load`: downloads the latest channel and stream data from the [iptv-org/api](https://github.com/iptv-org/api).
- `api:generate`: generates a JSON file with all streams for the [iptv-org/api](https://github.com/iptv-org/api) repository. - `api:generate`: generates a JSON file with all streams for the [iptv-org/api](https://github.com/iptv-org/api) repository.
- `api:deploy`: allows to manually upload a JSON file created via `api:generate` to the [iptv-org/api](https://github.com/iptv-org/api) repository. To run the script you must provide your [personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) with write access to the repository. - `api:deploy`: allows to manually upload a JSON file created via `api:generate` to the [iptv-org/api](https://github.com/iptv-org/api) repository. To run the script you must provide your [personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) with write access to the repository.
- `playlist:format`: formats internal playlists. The process includes [URL normalization](https://en.wikipedia.org/wiki/URI_normalization), duplicate removal, removing invalid id's and sorting links by channel name, quality, and label. - `playlist:format`: formats internal playlists. The process includes [URL normalization](https://en.wikipedia.org/wiki/URI_normalization), duplicate removal, removing invalid id's and sorting links by channel name, quality, and label.
- `playlist:update`: triggers an update of internal playlists. The process involves processing approved requests from issues. - `playlist:update`: triggers an update of internal playlists. The process involves processing approved requests from issues.
- `playlist:generate`: generates all public playlists. - `playlist:generate`: generates all public playlists.
- `playlist:validate`: сhecks ids and links in internal playlists for errors. - `playlist:validate`: сhecks ids and links in internal playlists for errors.
- `playlist:lint`: сhecks internal playlists for syntax errors. - `playlist:lint`: сhecks internal playlists for syntax errors.
- `playlist:test`: tests links in internal playlists. - `playlist:test`: tests links in internal playlists.
- `playlist:edit`: utility for quick streams mapping. - `playlist:edit`: utility for quick streams mapping.
- `playlist:deploy`: allows to manually publish all generated via `playlist:generate` playlists. To run the script you must provide your [personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) with write access to the repository. - `playlist:deploy`: allows to manually publish all generated via `playlist:generate` playlists. To run the script you must provide your [personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) with write access to the repository.
- `readme:update`: updates the list of playlists in [README.md](README.md). - `readme:update`: updates the list of playlists in [README.md](README.md).
- `report:create`: creates a report on current issues. - `report:create`: creates a report on current issues.
- `check`: (shorthand) sequentially runs the `playlist:lint` and `playlist:validate` scripts. - `check`: (shorthand) sequentially runs the `playlist:lint` and `playlist:validate` scripts.
- `format`: (shorthand) runs the `playlist:format` script. - `format`: (shorthand) runs the `playlist:format` script.
- `update`: (shorthand) sequentially runs the `playlist:generate`, `api:generate` and `readme:update` scripts. - `update`: (shorthand) sequentially runs the `playlist:generate`, `api:generate` and `readme:update` scripts.
- `deploy`: (shorthand) sequentially runs the `playlist:deploy` and `api:deploy` scripts. - `deploy`: (shorthand) sequentially runs the `playlist:deploy` and `api:deploy` scripts.
- `lint`: сhecks the scripts for syntax errors. - `lint`: сhecks the scripts for syntax errors.
- `test`: runs a test of all the scripts described above. - `test`: runs a test of all the scripts described above.
## Workflows ## Workflows
To automate the run of the scripts described above, we use the [GitHub Actions workflows](https://docs.github.com/en/actions/using-workflows). To automate the run of the scripts described above, we use the [GitHub Actions workflows](https://docs.github.com/en/actions/using-workflows).
Each workflow includes its own set of scripts that can be run either manually or in response to an event. Each workflow includes its own set of scripts that can be run either manually or in response to an event.
- `check`: sequentially runs the `api:load`, `playlist:check` and `playlist:validate` scripts when a new pull request appears, and blocks the merge if it detects an error in it. - `check`: sequentially runs the `api:load`, `playlist:check` and `playlist:validate` scripts when a new pull request appears, and blocks the merge if it detects an error in it.
- `format`: sequentially runs `api:load`, `playlist:format`, `playlist:lint` and `playlist:validate` scripts. - `format`: sequentially runs `api:load`, `playlist:format`, `playlist:lint` and `playlist:validate` scripts.
- `update`: every day at 0:00 UTC sequentially runs `api:load`, `playlist:update`, `playlist:lint`, `playlist:validate`, `playlist:generate`, `api:generate` and `readme:update` scripts and deploys the output files if successful. - `update`: every day at 0:00 UTC sequentially runs `api:load`, `playlist:update`, `playlist:lint`, `playlist:validate`, `playlist:generate`, `api:generate` and `readme:update` scripts and deploys the output files if successful.

46
FAQ.md
View File

@@ -1,23 +1,23 @@
# Frequently Asked Questions # Frequently Asked Questions
### My favorite channel is not on the playlist. ### My favorite channel is not on the playlist.
Start by asking our community for help via [Discussions](https://github.com/orgs/iptv-org/discussions). It is quite possible that someone already has a link to the channel you need and they just haven't added it to our playlist yet. Start by asking our community for help via [Discussions](https://github.com/orgs/iptv-org/discussions). It is quite possible that someone already has a link to the channel you need and they just haven't added it to our playlist yet.
But keep in mind that not all TV channels are available for viewing online, and in this case there is little we can do about it. But keep in mind that not all TV channels are available for viewing online, and in this case there is little we can do about it.
### Are you planning to include a Video On Demand (VOD) to the playlist? ### Are you planning to include a Video On Demand (VOD) to the playlist?
No. No.
### Why is the channel on the iptv-org.github.io but not in the playlist? ### Why is the channel on the iptv-org.github.io but not in the playlist?
The site contains a list of all TV channels in the world and only those of them for which we have working stream links are included in the playlists. The site contains a list of all TV channels in the world and only those of them for which we have working stream links are included in the playlists.
### Can I add a radio broadcast? ### Can I add a radio broadcast?
Yes, if it is a [visual radio](https://en.wikipedia.org/wiki/Visual_radio) in which a video and audio are shown at the same time. Yes, if it is a [visual radio](https://en.wikipedia.org/wiki/Visual_radio) in which a video and audio are shown at the same time.
### Why don't you accept links to Xtream Codes server? ### Why don't you accept links to Xtream Codes server?
Xtream Codes streams tend to be very unstable, and often links to them fail very quickly, so it's easier for us to initially exclude them from the playlist than to search for expired ones every day. Xtream Codes streams tend to be very unstable, and often links to them fail very quickly, so it's easier for us to initially exclude them from the playlist than to search for expired ones every day.

48
LICENSE
View File

@@ -1,24 +1,24 @@
This is free and unencumbered software released into the public domain. This is free and unencumbered software released into the public domain.
Anyone is free to copy, modify, publish, use, compile, sell, or Anyone is free to copy, modify, publish, use, compile, sell, or
distribute this software, either in source code form or as a compiled distribute this software, either in source code form or as a compiled
binary, for any purpose, commercial or non-commercial, and by any binary, for any purpose, commercial or non-commercial, and by any
means. means.
In jurisdictions that recognize copyright laws, the author or authors In jurisdictions that recognize copyright laws, the author or authors
of this software dedicate any and all copyright interest in the of this software dedicate any and all copyright interest in the
software to the public domain. We make this dedication for the benefit software to the public domain. We make this dedication for the benefit
of the public at large and to the detriment of our heirs and of the public at large and to the detriment of our heirs and
successors. We intend this dedication to be an overt act of successors. We intend this dedication to be an overt act of
relinquishment in perpetuity of all present and future rights to this relinquishment in perpetuity of all present and future rights to this
software under copyright law. software under copyright law.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE. OTHER DEALINGS IN THE SOFTWARE.
For more information, please refer to <http://unlicense.org/> For more information, please refer to <http://unlicense.org/>

176
PLAYLISTS.md
View File

@@ -1,22 +1,22 @@
## Playlists ## Playlists
There are several versions of playlists that differ in the way they are grouped. As of January 30th, 2024, we have stopped distributing NSFW channels. For more information, please look at [this issue](https://github.com/iptv-org/iptv/issues/15723). There are several versions of playlists that differ in the way they are grouped. As of January 30th, 2024, we have stopped distributing NSFW channels. For more information, please look at [this issue](https://github.com/iptv-org/iptv/issues/15723).
### Grouped by category ### Grouped by category
Playlists in which channels are grouped by category. Playlists in which channels are grouped by category.
<details> <details>
<summary>Expand</summary> <summary>Expand</summary>
<br> <br>
``` ```
https://iptv-org.github.io/iptv/index.category.m3u https://iptv-org.github.io/iptv/index.category.m3u
``` ```
Same thing, but split up into separate files: Same thing, but split up into separate files:
<!-- prettier-ignore --> <!-- prettier-ignore -->
<table> <table>
<thead> <thead>
<tr><th align="left">Category</th><th align="left">Channels</th><th align="left">Playlist</th></tr> <tr><th align="left">Category</th><th align="left">Channels</th><th align="left">Playlist</th></tr>
@@ -54,25 +54,25 @@ Same thing, but split up into separate files:
<tr><td>XXX</td><td align="right">0</td><td nowrap><code>https://iptv-org.github.io/iptv/categories/xxx.m3u</code></td></tr> <tr><td>XXX</td><td align="right">0</td><td nowrap><code>https://iptv-org.github.io/iptv/categories/xxx.m3u</code></td></tr>
<tr><td>Undefined</td><td align="right">3696</td><td nowrap><code>https://iptv-org.github.io/iptv/categories/undefined.m3u</code></td></tr> <tr><td>Undefined</td><td align="right">3696</td><td nowrap><code>https://iptv-org.github.io/iptv/categories/undefined.m3u</code></td></tr>
</tbody> </tbody>
</table> </table>
</details> </details>
### Grouped by language ### Grouped by language
Playlists in which channels are grouped by the language in which they are broadcast. Playlists in which channels are grouped by the language in which they are broadcast.
<details> <details>
<summary>Expand</summary> <summary>Expand</summary>
<br> <br>
``` ```
https://iptv-org.github.io/iptv/index.language.m3u https://iptv-org.github.io/iptv/index.language.m3u
``` ```
Same thing, but split up into separate files: Same thing, but split up into separate files:
<!-- prettier-ignore --> <!-- prettier-ignore -->
<table> <table>
<thead> <thead>
<tr><th align="left">Language</th><th align="left">Channels</th><th align="left">Playlist</th></tr> <tr><th align="left">Language</th><th align="left">Channels</th><th align="left">Playlist</th></tr>
@@ -292,26 +292,26 @@ Same thing, but split up into separate files:
<tr><td align="left">Zulu</td><td align="right">1</td><td align="left" nowrap><code>https://iptv-org.github.io/iptv/languages/zul.m3u</code></td></tr> <tr><td align="left">Zulu</td><td align="right">1</td><td align="left" nowrap><code>https://iptv-org.github.io/iptv/languages/zul.m3u</code></td></tr>
<tr><td align="left">Undefined</td><td align="right">2176</td><td align="left" nowrap><code>https://iptv-org.github.io/iptv/languages/undefined.m3u</code></td></tr> <tr><td align="left">Undefined</td><td align="right">2176</td><td align="left" nowrap><code>https://iptv-org.github.io/iptv/languages/undefined.m3u</code></td></tr>
</tbody> </tbody>
</table> </table>
</details> </details>
### Grouped by broadcast area ### Grouped by broadcast area
Playlists in which channels are grouped by broadcast area. Playlists in which channels are grouped by broadcast area.
<details> <details>
<summary>Expand</summary> <summary>Expand</summary>
#### Countries #### Countries
``` ```
https://iptv-org.github.io/iptv/index.country.m3u https://iptv-org.github.io/iptv/index.country.m3u
``` ```
Same thing, but split up into separate files: Same thing, but split up into separate files:
<!-- prettier-ignore --> <!-- prettier-ignore -->
- 🇦🇫 Afghanistan <code>https://iptv-org.github.io/iptv/countries/af.m3u</code> - 🇦🇫 Afghanistan <code>https://iptv-org.github.io/iptv/countries/af.m3u</code>
- 🇦🇱 Albania <code>https://iptv-org.github.io/iptv/countries/al.m3u</code> - 🇦🇱 Albania <code>https://iptv-org.github.io/iptv/countries/al.m3u</code>
- 🇩🇿 Algeria <code>https://iptv-org.github.io/iptv/countries/dz.m3u</code> - 🇩🇿 Algeria <code>https://iptv-org.github.io/iptv/countries/dz.m3u</code>
@@ -1293,11 +1293,11 @@ Same thing, but split up into separate files:
- 🇿🇲 Zambia <code>https://iptv-org.github.io/iptv/countries/zm.m3u</code> - 🇿🇲 Zambia <code>https://iptv-org.github.io/iptv/countries/zm.m3u</code>
- 🇿🇼 Zimbabwe <code>https://iptv-org.github.io/iptv/countries/zw.m3u</code> - 🇿🇼 Zimbabwe <code>https://iptv-org.github.io/iptv/countries/zw.m3u</code>
- 🌐 International <code>https://iptv-org.github.io/iptv/countries/int.m3u</code> - 🌐 International <code>https://iptv-org.github.io/iptv/countries/int.m3u</code>
- Undefined <code>https://iptv-org.github.io/iptv/countries/undefined.m3u</code> - Undefined <code>https://iptv-org.github.io/iptv/countries/undefined.m3u</code>
#### Regions #### Regions
<!-- prettier-ignore --> <!-- prettier-ignore -->
- Africa <code>https://iptv-org.github.io/iptv/regions/afr.m3u</code> - Africa <code>https://iptv-org.github.io/iptv/regions/afr.m3u</code>
- Americas <code>https://iptv-org.github.io/iptv/regions/amer.m3u</code> - Americas <code>https://iptv-org.github.io/iptv/regions/amer.m3u</code>
- Arab world <code>https://iptv-org.github.io/iptv/regions/arab.m3u</code> - Arab world <code>https://iptv-org.github.io/iptv/regions/arab.m3u</code>
@@ -1339,28 +1339,28 @@ Same thing, but split up into separate files:
- West Africa <code>https://iptv-org.github.io/iptv/regions/waf.m3u</code> - West Africa <code>https://iptv-org.github.io/iptv/regions/waf.m3u</code>
- West Asia <code>https://iptv-org.github.io/iptv/regions/was.m3u</code> - West Asia <code>https://iptv-org.github.io/iptv/regions/was.m3u</code>
- Western Europe <code>https://iptv-org.github.io/iptv/regions/wer.m3u</code> - Western Europe <code>https://iptv-org.github.io/iptv/regions/wer.m3u</code>
- Worldwide <code>https://iptv-org.github.io/iptv/regions/ww.m3u</code> - Worldwide <code>https://iptv-org.github.io/iptv/regions/ww.m3u</code>
</details> </details>
### Grouped by sources ### Grouped by sources
Playlists in which channels are grouped by broadcast source. Playlists in which channels are grouped by broadcast source.
<details> <details>
<summary>Expand</summary> <summary>Expand</summary>
<br> <br>
To use the playlist, simply replace `<FILENAME>` in the link below with the name of one of the files in the [streams](streams) folder. To use the playlist, simply replace `<FILENAME>` in the link below with the name of one of the files in the [streams](streams) folder.
``` ```
https://iptv-org.github.io/iptv/sources/<FILENAME>.m3u https://iptv-org.github.io/iptv/sources/<FILENAME>.m3u
``` ```
</details> </details>
Also, any of our internal playlists are available in raw form (without any filtering or sorting) at this link: Also, any of our internal playlists are available in raw form (without any filtering or sorting) at this link:
``` ```
https://iptv-org.github.io/iptv/raw/<FILENAME>.m3u https://iptv-org.github.io/iptv/raw/<FILENAME>.m3u
``` ```

158
README.md
View File

@@ -1,79 +1,79 @@
# IPTV [![update](https://github.com/iptv-org/iptv/actions/workflows/update.yml/badge.svg)](https://github.com/iptv-org/iptv/actions/workflows/update.yml) # IPTV [![update](https://github.com/iptv-org/iptv/actions/workflows/update.yml/badge.svg)](https://github.com/iptv-org/iptv/actions/workflows/update.yml)
Collection of publicly available IPTV (Internet Protocol television) channels from all over the world. Collection of publicly available IPTV (Internet Protocol television) channels from all over the world.
## Table of contents ## Table of contents
- 🚀 [How to use?](#how-to-use) - 🚀 [How to use?](#how-to-use)
- 📺 [Playlists](#playlists) - 📺 [Playlists](#playlists)
- 🗓 [EPG](#epg) - 🗓 [EPG](#epg)
- 🗄 [Database](#database) - 🗄 [Database](#database)
- 👨‍💻 [API](#api) - 👨‍💻 [API](#api)
- 📚 [Resources](#resources) - 📚 [Resources](#resources)
- 💬 [Discussions](#discussions) - 💬 [Discussions](#discussions)
- ❓ [FAQ](#faq) - ❓ [FAQ](#faq)
- 🛠 [Contribution](#contribution) - 🛠 [Contribution](#contribution)
- ⚖ [Legal](#legal) - ⚖ [Legal](#legal)
- © [License](#license) - © [License](#license)
## How to use? ## How to use?
Simply paste the link to one of the playlists into [any video player](https://github.com/iptv-org/awesome-iptv#apps) that supports live streaming and press _Open_. Simply paste the link to one of the playlists into [any video player](https://github.com/iptv-org/awesome-iptv#apps) that supports live streaming and press _Open_.
![VLC Network Panel](https://github.com/iptv-org/iptv/raw/master/.readme/preview.png) ![VLC Network Panel](https://github.com/iptv-org/iptv/raw/master/.readme/preview.png)
## Playlists ## Playlists
The main playlist containing all channels available in the repository can be found at: The main playlist containing all channels available in the repository can be found at:
``` ```
https://iptv-org.github.io/iptv/index.m3u https://iptv-org.github.io/iptv/index.m3u
``` ```
Links to other playlists can be found in the [PLAYLISTS.md](PLAYLISTS.md) file. Links to other playlists can be found in the [PLAYLISTS.md](PLAYLISTS.md) file.
## EPG ## EPG
[Electronic Program Guide](https://en.wikipedia.org/wiki/Electronic_program_guide) for most of the channels can be downloaded using utilities published in the [iptv-org/epg](https://github.com/iptv-org/epg) repository. [Electronic Program Guide](https://en.wikipedia.org/wiki/Electronic_program_guide) for most of the channels can be downloaded using utilities published in the [iptv-org/epg](https://github.com/iptv-org/epg) repository.
## Database ## Database
All channel data is taken from the [iptv-org/database](https://github.com/iptv-org/database) repository. If you find any errors please open a new [issue](https://github.com/iptv-org/database/issues) there. All channel data is taken from the [iptv-org/database](https://github.com/iptv-org/database) repository. If you find any errors please open a new [issue](https://github.com/iptv-org/database/issues) there.
## API ## API
The API documentation can be found in the [iptv-org/api](https://github.com/iptv-org/api) repository. The API documentation can be found in the [iptv-org/api](https://github.com/iptv-org/api) repository.
## Resources ## Resources
Links to other useful IPTV-related resources can be found in the [iptv-org/awesome-iptv](https://github.com/iptv-org/awesome-iptv) repository. Links to other useful IPTV-related resources can be found in the [iptv-org/awesome-iptv](https://github.com/iptv-org/awesome-iptv) repository.
## Discussions ## Discussions
If you need help finding a channel, have a question or idea, welcome to the [Discussions](https://github.com/orgs/iptv-org/discussions). If you need help finding a channel, have a question or idea, welcome to the [Discussions](https://github.com/orgs/iptv-org/discussions).
## FAQ ## FAQ
The answers to the most popular questions can be found in the [FAQ.md](FAQ.md) file. The answers to the most popular questions can be found in the [FAQ.md](FAQ.md) file.
## Contribution ## Contribution
Please make sure to read the [Contributing Guide](CONTRIBUTING.md) before sending an issue or making a pull request. Please make sure to read the [Contributing Guide](CONTRIBUTING.md) before sending an issue or making a pull request.
And thank you to everyone who has already contributed! And thank you to everyone who has already contributed!
### Backers ### Backers
<a href="https://opencollective.com/iptv-org"><img src="https://opencollective.com/iptv-org/backers.svg?width=890" /></a> <a href="https://opencollective.com/iptv-org"><img src="https://opencollective.com/iptv-org/backers.svg?width=890" /></a>
### Contributors ### Contributors
<a href="https://github.com/iptv-org/iptv/graphs/contributors"><img src="https://opencollective.com/iptv-org/contributors.svg?width=890" /></a> <a href="https://github.com/iptv-org/iptv/graphs/contributors"><img src="https://opencollective.com/iptv-org/contributors.svg?width=890" /></a>
## Legal ## Legal
No video files are stored in this repository. The repository simply contains user-submitted links to publicly available video stream URLs, which to the best of our knowledge have been intentionally made publicly by the copyright holders. If any links in these playlists infringe on your rights as a copyright holder, they may be removed by sending a [pull request](https://github.com/iptv-org/iptv/pulls) or opening an [issue](https://github.com/iptv-org/iptv/issues/new?assignees=freearhey&labels=removal+request&template=--removal-request.yml&title=Remove%3A+). However, note that we have **no control** over the destination of the link, and just removing the link from the playlist will not remove its contents from the web. Note that linking does not directly infringe copyright because no copy is made on the site providing the link, and thus this is **not** a valid reason to send a DMCA notice to GitHub. To remove this content from the web, you should contact the web host that's actually hosting the content (**not** GitHub, nor the maintainers of this repository). No video files are stored in this repository. The repository simply contains user-submitted links to publicly available video stream URLs, which to the best of our knowledge have been intentionally made publicly by the copyright holders. If any links in these playlists infringe on your rights as a copyright holder, they may be removed by sending a [pull request](https://github.com/iptv-org/iptv/pulls) or opening an [issue](https://github.com/iptv-org/iptv/issues/new?assignees=freearhey&labels=removal+request&template=--removal-request.yml&title=Remove%3A+). However, note that we have **no control** over the destination of the link, and just removing the link from the playlist will not remove its contents from the web. Note that linking does not directly infringe copyright because no copy is made on the site providing the link, and thus this is **not** a valid reason to send a DMCA notice to GitHub. To remove this content from the web, you should contact the web host that's actually hosting the content (**not** GitHub, nor the maintainers of this repository).
## License ## License
[![CC0](http://mirrors.creativecommons.org/presskit/buttons/88x31/svg/cc-zero.svg)](LICENSE) [![CC0](http://mirrors.creativecommons.org/presskit/buttons/88x31/svg/cc-zero.svg)](LICENSE)

112
eslint.config.mjs
View File

@@ -1,56 +1,56 @@
import typescriptEslint from '@typescript-eslint/eslint-plugin' import typescriptEslint from '@typescript-eslint/eslint-plugin'
import globals from 'globals' import stylistic from '@stylistic/eslint-plugin'
import tsParser from '@typescript-eslint/parser' import tsParser from '@typescript-eslint/parser'
import path from 'node:path' import { FlatCompat } from '@eslint/eslintrc'
import { fileURLToPath } from 'node:url' import { fileURLToPath } from 'node:url'
import js from '@eslint/js' import globals from 'globals'
import stylistic from '@stylistic/eslint-plugin' import path from 'node:path'
import { FlatCompat } from '@eslint/eslintrc' import js from '@eslint/js'
const __filename = fileURLToPath(import.meta.url) const __filename = fileURLToPath(import.meta.url)
const __dirname = path.dirname(__filename) const __dirname = path.dirname(__filename)
const compat = new FlatCompat({ const compat = new FlatCompat({
baseDirectory: __dirname, baseDirectory: __dirname,
recommendedConfig: js.configs.recommended, recommendedConfig: js.configs.recommended,
allConfig: js.configs.all allConfig: js.configs.all
}) })
export default [ export default [
...compat.extends('eslint:recommended', 'plugin:@typescript-eslint/recommended'), ...compat.extends('eslint:recommended', 'plugin:@typescript-eslint/recommended'),
{ {
plugins: { plugins: {
'@typescript-eslint': typescriptEslint, '@typescript-eslint': typescriptEslint,
'@stylistic': stylistic '@stylistic': stylistic
}, },
languageOptions: { languageOptions: {
globals: { globals: {
...globals.browser ...globals.browser
}, },
parser: tsParser, parser: tsParser,
ecmaVersion: 'latest', ecmaVersion: 'latest',
sourceType: 'module' sourceType: 'module'
}, },
rules: { rules: {
'no-case-declarations': 'off', 'no-case-declarations': 'off',
indent: [ indent: [
'error', 'error',
2, 2,
{ {
SwitchCase: 1 SwitchCase: 1
} }
], ],
'@stylistic/linebreak-style': ['error', 'windows'], '@stylistic/linebreak-style': ['error', 'windows'],
quotes: ['error', 'single'], quotes: ['error', 'single'],
semi: ['error', 'never'] semi: ['error', 'never']
} }
}, },
{ {
ignores: ['tests/__data__/**'] ignores: ['tests/__data__/**']
} }
] ]

34
m3u-linter.json
View File

@@ -1,17 +1,17 @@
{ {
"files": ["streams/*.m3u"], "files": ["streams/*.m3u"],
"rules": { "rules": {
"no-empty-lines": true, "no-empty-lines": true,
"require-header": true, "require-header": true,
"attribute-quotes": true, "attribute-quotes": true,
"require-info": true, "require-info": true,
"require-title": true, "require-title": true,
"no-trailing-spaces": false, "no-trailing-spaces": false,
"no-whitespace-before-title": true, "no-whitespace-before-title": true,
"no-multi-spaces": true, "no-multi-spaces": true,
"no-extra-comma": true, "no-extra-comma": true,
"space-before-paren": true, "space-before-paren": true,
"no-dash": true, "no-dash": true,
"require-link": true "require-link": true
} }
} }

15752
package-lock.json generated
View File

File diff suppressed because it is too large Load Diff

164
package.json
View File

@@ -1,80 +1,84 @@
{ {
"name": "iptv", "name": "iptv",
"scripts": { "scripts": {
"act:check": "act pull_request -W .github/workflows/check.yml", "act:check": "act pull_request -W .github/workflows/check.yml -s GITHUB_TOKEN=\"$(gh auth token)\"",
"act:format": "act workflow_dispatch -W .github/workflows/format.yml", "act:format": "act workflow_dispatch -W .github/workflows/format.yml -s GITHUB_TOKEN=\"$(gh auth token)\"",
"act:update": "act workflow_dispatch -W .github/workflows/update.yml", "act:update": "act workflow_dispatch -W .github/workflows/update.yml -s GITHUB_TOKEN=\"$(gh auth token)\"",
"api:load": "tsx scripts/commands/api/load.ts", "api:load": "tsx scripts/commands/api/load.ts",
"api:generate": "tsx scripts/commands/api/generate.ts", "api:generate": "tsx scripts/commands/api/generate.ts",
"api:deploy": "npx gh-pages-clean && npx gh-pages -a -m \"Deploy to iptv-org/api\" -d .api -r https://$GITHUB_TOKEN@github.com/iptv-org/api.git", "api:deploy": "npx gh-pages-clean && npx gh-pages -a -m \"Deploy to iptv-org/api\" -d .api -r https://$GITHUB_TOKEN@github.com/iptv-org/api.git",
"playlist:format": "tsx scripts/commands/playlist/format.ts", "playlist:format": "tsx scripts/commands/playlist/format.ts",
"playlist:update": "tsx scripts/commands/playlist/update.ts", "playlist:update": "tsx scripts/commands/playlist/update.ts",
"playlist:generate": "tsx scripts/commands/playlist/generate.ts", "playlist:generate": "tsx scripts/commands/playlist/generate.ts",
"playlist:validate": "tsx scripts/commands/playlist/validate.ts", "playlist:validate": "tsx scripts/commands/playlist/validate.ts",
"playlist:lint": "npx m3u-linter -c m3u-linter.json", "playlist:lint": "npx m3u-linter -c m3u-linter.json",
"playlist:test": "tsx scripts/commands/playlist/test.ts", "playlist:test": "tsx scripts/commands/playlist/test.ts",
"playlist:edit": "tsx scripts/commands/playlist/edit.ts", "playlist:edit": "tsx scripts/commands/playlist/edit.ts",
"playlist:deploy": "npx gh-pages-clean && npx gh-pages -m \"Deploy to GitHub Pages\" -d .gh-pages -r https://$GITHUB_TOKEN@github.com/iptv-org/iptv.git", "playlist:deploy": "npx gh-pages-clean && npx gh-pages -m \"Deploy to GitHub Pages\" -d .gh-pages -r https://$GITHUB_TOKEN@github.com/iptv-org/iptv.git",
"readme:update": "tsx scripts/commands/readme/update.ts", "readme:update": "tsx scripts/commands/readme/update.ts",
"report:create": "tsx scripts/commands/report/create.ts", "report:create": "tsx scripts/commands/report/create.ts",
"check": "npm run playlist:lint && npm run playlist:validate", "check": "npm run playlist:lint && npm run playlist:validate",
"format": "npm run playlist:format", "format": "npm run playlist:format",
"update": "npm run playlist:generate && npm run api:generate && npm run readme:update", "update": "npm run playlist:generate && npm run api:generate && npm run readme:update",
"deploy": "npm run playlist:deploy && npm run api:deploy", "deploy": "npm run playlist:deploy && npm run api:deploy",
"lint": "npx eslint \"scripts/**/*.{ts,js}\" \"tests/**/*.{ts,js}\"", "lint": "npx eslint \"scripts/**/*.{ts,js}\" \"tests/**/*.{ts,js}\"",
"test": "jest --runInBand", "test": "jest --runInBand",
"postinstall": "npm run api:load" "postinstall": "npm run api:load"
}, },
"jest": { "jest": {
"transform": { "transform": {
"^.+\\.ts$": "@swc/jest" "^.+\\.ts$": "@swc/jest"
}, },
"testRegex": "tests/(.*?/)?.*test.ts$", "testRegex": "tests/(.*?/)?.*test.ts$",
"setupFilesAfterEnv": [ "setupFilesAfterEnv": [
"jest-expect-message" "jest-expect-message"
] ]
}, },
"author": "Arhey", "author": "Arhey",
"private": true, "private": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@alex_neo/jest-expect-message": "^1.0.5", "@alex_neo/jest-expect-message": "^1.0.5",
"@eslint/eslintrc": "^3.3.1", "@eslint/eslintrc": "^3.3.1",
"@eslint/js": "^9.32.0", "@eslint/js": "^9.32.0",
"@freearhey/core": "^0.10.2", "@freearhey/core": "^0.14.3",
"@freearhey/search-js": "^0.1.2", "@freearhey/search-js": "^0.1.2",
"@inquirer/prompts": "^7.8.0", "@freearhey/storage-js": "^0.1.0",
"@octokit/core": "^7.0.3", "@inquirer/prompts": "^7.8.0",
"@octokit/plugin-paginate-rest": "^13.1.1", "@iptv-org/sdk": "^1.0.2",
"@octokit/plugin-rest-endpoint-methods": "^16.0.0", "@octokit/core": "^7.0.3",
"@octokit/types": "^14.1.0", "@octokit/plugin-paginate-rest": "^13.1.1",
"@stylistic/eslint-plugin": "^5.2.2", "@octokit/plugin-rest-endpoint-methods": "^16.0.0",
"@swc/jest": "^0.2.39", "@octokit/types": "^14.1.0",
"@types/async": "^3.2.25", "@stylistic/eslint-plugin": "^5.2.2",
"@types/cli-progress": "^3.11.6", "@swc/jest": "^0.2.39",
"@types/fs-extra": "^11.0.4", "@types/async": "^3.2.25",
"@types/jest": "^30.0.0", "@types/cli-progress": "^3.11.6",
"@types/lodash.uniqueid": "^4.0.9", "@types/fs-extra": "^11.0.4",
"@typescript-eslint/eslint-plugin": "^8.38.0", "@types/jest": "^30.0.0",
"@typescript-eslint/parser": "^8.38.0", "@types/lodash.uniqueid": "^4.0.9",
"async-es": "^3.2.6", "@types/node-cleanup": "^2.1.5",
"axios": "^1.11.0", "@typescript-eslint/eslint-plugin": "^8.38.0",
"chalk": "^5.4.1", "@typescript-eslint/parser": "^8.38.0",
"cli-progress": "^3.12.0", "async": "^3.2.6",
"commander": "^14.0.0", "axios": "^1.11.0",
"console-table-printer": "^2.14.6", "chalk": "^5.4.1",
"cross-env": "^10.0.0", "cli-progress": "^3.12.0",
"eslint": "^9.32.0", "commander": "^14.0.0",
"glob": "^11.0.3", "console-table-printer": "^2.14.6",
"globals": "^16.3.0", "cross-env": "^10.0.0",
"iptv-playlist-parser": "^0.15.0", "eslint": "^9.32.0",
"jest": "^30.0.5", "glob": "^11.0.3",
"jest-expect-message": "^1.1.3", "globals": "^16.3.0",
"lodash.uniqueid": "^4.0.1", "iptv-playlist-parser": "^0.15.1",
"m3u-linter": "^0.4.2", "jest": "^30.0.5",
"mediainfo.js": "^0.3.6", "jest-expect-message": "^1.1.3",
"node-cleanup": "^2.1.2", "lodash.uniqueid": "^4.0.1",
"socks-proxy-agent": "^8.0.5", "m3u-linter": "^0.4.2",
"tsx": "^4.20.3" "mediainfo.js": "^0.3.6",
} "node-cleanup": "^2.1.2",
} "normalize-url": "^8.1.0",
"socks-proxy-agent": "^8.0.5",
"tsx": "^4.20.3"
}
}

151
scripts/api.ts Normal file
View File

@@ -0,0 +1,151 @@
import { Collection, Dictionary } from '@freearhey/core'
import { DATA_DIR } from './constants'
import cliProgress from 'cli-progress'
import * as sdk from '@iptv-org/sdk'
const data = {
categoriesKeyById: new Dictionary<sdk.Models.Category>(),
countriesKeyByCode: new Dictionary<sdk.Models.Country>(),
subdivisionsKeyByCode: new Dictionary<sdk.Models.Subdivision>(),
citiesKeyByCode: new Dictionary<sdk.Models.City>(),
regionsKeyByCode: new Dictionary<sdk.Models.Region>(),
languagesKeyByCode: new Dictionary<sdk.Models.Language>(),
channelsKeyById: new Dictionary<sdk.Models.Channel>(),
feedsKeyByStreamId: new Dictionary<sdk.Models.Feed>(),
feedsGroupedByChannel: new Dictionary<sdk.Models.Feed[]>(),
blocklistRecordsGroupedByChannel: new Dictionary<sdk.Models.BlocklistRecord[]>(),
categories: new Collection<sdk.Models.Category>(),
countries: new Collection<sdk.Models.Country>(),
subdivisions: new Collection<sdk.Models.Subdivision>(),
cities: new Collection<sdk.Models.City>(),
regions: new Collection<sdk.Models.Region>()
}
let searchIndex
async function loadData() {
const dataManager = new sdk.DataManager({ dataDir: DATA_DIR })
await dataManager.loadFromDisk()
dataManager.processData()
const {
channels,
feeds,
categories,
languages,
countries,
subdivisions,
cities,
regions,
blocklist
} = dataManager.getProcessedData()
searchIndex = sdk.SearchEngine.createIndex<sdk.Models.Channel>(channels)
data.categoriesKeyById = categories.keyBy((category: sdk.Models.Category) => category.id)
data.countriesKeyByCode = countries.keyBy((country: sdk.Models.Country) => country.code)
data.subdivisionsKeyByCode = subdivisions.keyBy(
(subdivision: sdk.Models.Subdivision) => subdivision.code
)
data.citiesKeyByCode = cities.keyBy((city: sdk.Models.City) => city.code)
data.regionsKeyByCode = regions.keyBy((region: sdk.Models.Region) => region.code)
data.languagesKeyByCode = languages.keyBy((language: sdk.Models.Language) => language.code)
data.channelsKeyById = channels.keyBy((channel: sdk.Models.Channel) => channel.id)
data.feedsKeyByStreamId = feeds.keyBy((feed: sdk.Models.Feed) => feed.getStreamId())
data.feedsGroupedByChannel = feeds.groupBy((feed: sdk.Models.Feed) => feed.channel)
data.blocklistRecordsGroupedByChannel = blocklist.groupBy(
(blocklistRecord: sdk.Models.BlocklistRecord) => blocklistRecord.channel
)
data.categories = categories
data.countries = countries
data.subdivisions = subdivisions
data.cities = cities
data.regions = regions
}
async function downloadData() {
function formatBytes(bytes: number) {
if (bytes === 0) return '0 B'
const k = 1024
const sizes = ['B', 'KB', 'MB', 'GB']
const i = Math.floor(Math.log(bytes) / Math.log(k))
return parseFloat((bytes / Math.pow(k, i)).toFixed(1)) + ' ' + sizes[i]
}
const files = [
'blocklist',
'categories',
'channels',
'cities',
'countries',
'feeds',
'guides',
'languages',
'logos',
'regions',
'streams',
'subdivisions',
'timezones'
]
const multiBar = new cliProgress.MultiBar({
stopOnComplete: true,
hideCursor: true,
forceRedraw: true,
barsize: 36,
format(options, params, payload) {
const filename = payload.filename.padEnd(18, ' ')
const barsize = options.barsize || 40
const percent = (params.progress * 100).toFixed(2)
const speed = payload.speed ? formatBytes(payload.speed) + '/s' : 'N/A'
const total = formatBytes(params.total)
const completeSize = Math.round(params.progress * barsize)
const incompleteSize = barsize - completeSize
const bar =
options.barCompleteString && options.barIncompleteString
? options.barCompleteString.substr(0, completeSize) +
options.barGlue +
options.barIncompleteString.substr(0, incompleteSize)
: '-'.repeat(barsize)
return `${filename} [${bar}] ${percent}% | ETA: ${params.eta}s | ${total} | ${speed}`
}
})
const dataManager = new sdk.DataManager({ dataDir: DATA_DIR })
const requests: Promise<unknown>[] = []
for (const basename of files) {
const filename = `${basename}.json`
const progressBar = multiBar.create(0, 0, { filename })
const request = dataManager.downloadFileToDisk(basename, {
onDownloadProgress({ total, loaded, rate }) {
if (total) progressBar.setTotal(total)
progressBar.update(loaded, { speed: rate })
}
})
requests.push(request)
}
await Promise.allSettled(requests).catch(console.error)
}
function searchChannels(query: string): Collection<sdk.Models.Channel> {
if (!searchIndex) return new Collection<sdk.Models.Channel>()
const results = searchIndex.search(query)
const channels = new Collection<sdk.Models.Channel>()
new Collection<sdk.Types.ChannelSearchableData>(results).forEach(
(item: sdk.Types.ChannelSearchableData) => {
const channel = data.channelsKeyById.get(item.id)
if (channel) channels.add(channel)
}
)
return channels
}
export { data, loadData, downloadData, searchChannels }

70
scripts/commands/api/generate.ts
View File

@@ -1,39 +1,31 @@
import { DataLoader, DataProcessor, PlaylistParser } from '../../core' import { API_DIR, STREAMS_DIR } from '../../constants'
import type { DataProcessorData } from '../../types/dataProcessor' import { Storage } from '@freearhey/storage-js'
import { API_DIR, STREAMS_DIR, DATA_DIR } from '../../constants' import { PlaylistParser } from '../../core'
import type { DataLoaderData } from '../../types/dataLoader' import { Logger } from '@freearhey/core'
import { Logger, Storage } from '@freearhey/core' import { Stream } from '../../models'
import { Stream } from '../../models' import { loadData } from '../../api'
async function main() { async function main() {
const logger = new Logger() const logger = new Logger()
logger.info('loading data from api...') logger.info('loading data from api...')
const processor = new DataProcessor() await loadData()
const dataStorage = new Storage(DATA_DIR)
const dataLoader = new DataLoader({ storage: dataStorage }) logger.info('loading streams...')
const data: DataLoaderData = await dataLoader.load() const streamsStorage = new Storage(STREAMS_DIR)
const { channelsKeyById, feedsGroupedByChannelId, logosGroupedByStreamId }: DataProcessorData = const parser = new PlaylistParser({
processor.process(data) storage: streamsStorage
})
logger.info('loading streams...') const files = await streamsStorage.list('**/*.m3u')
const streamsStorage = new Storage(STREAMS_DIR) const parsed = await parser.parse(files)
const parser = new PlaylistParser({ const _streams = parsed
storage: streamsStorage, .sortBy((stream: Stream) => stream.getId())
channelsKeyById, .map((stream: Stream) => stream.toObject())
logosGroupedByStreamId, logger.info(`found ${_streams.count()} streams`)
feedsGroupedByChannelId
}) logger.info('saving to .api/streams.json...')
const files = await streamsStorage.list('**/*.m3u') const apiStorage = new Storage(API_DIR)
let streams = await parser.parse(files) await apiStorage.save('streams.json', _streams.toJSON())
streams = streams }
.orderBy((stream: Stream) => stream.getId())
.map((stream: Stream) => stream.toJSON()) main()
logger.info(`found ${streams.count()} streams`)
logger.info('saving to .api/streams.json...')
const apiStorage = new Storage(API_DIR)
await apiStorage.save('streams.json', streams.toJSON())
}
main()

33
scripts/commands/api/load.ts
View File

@@ -1,26 +1,7 @@
import { DATA_DIR } from '../../constants' import { downloadData } from '../../api'
import { Storage } from '@freearhey/core'
import { DataLoader } from '../../core' async function main() {
await downloadData()
async function main() { }
const storage = new Storage(DATA_DIR)
const loader = new DataLoader({ storage }) main()
await Promise.all([
loader.download('blocklist.json'),
loader.download('categories.json'),
loader.download('channels.json'),
loader.download('countries.json'),
loader.download('languages.json'),
loader.download('regions.json'),
loader.download('subdivisions.json'),
loader.download('feeds.json'),
loader.download('logos.json'),
loader.download('timezones.json'),
loader.download('guides.json'),
loader.download('streams.json'),
loader.download('cities.json')
])
}
main()

407
scripts/commands/playlist/edit.ts
View File

@@ -1,217 +1,190 @@
import { Storage, Collection, Logger, Dictionary } from '@freearhey/core' import { loadData, data, searchChannels } from '../../api'
import { DataLoader, DataProcessor, PlaylistParser } from '../../core' import { Collection, Logger } from '@freearhey/core'
import type { ChannelSearchableData } from '../../types/channel' import { select, input } from '@inquirer/prompts'
import { Channel, Feed, Playlist, Stream } from '../../models' import { Playlist, Stream } from '../../models'
import { DataProcessorData } from '../../types/dataProcessor' import { Storage } from '@freearhey/storage-js'
import { DataLoaderData } from '../../types/dataLoader' import { PlaylistParser } from '../../core'
import { select, input } from '@inquirer/prompts' import nodeCleanup from 'node-cleanup'
import { DATA_DIR } from '../../constants' import * as sdk from '@iptv-org/sdk'
import nodeCleanup from 'node-cleanup' import { truncate } from '../../utils'
import sjs from '@freearhey/search-js' import { Command } from 'commander'
import { Command } from 'commander' import readline from 'readline'
import readline from 'readline'
type ChoiceValue = { type: string; value?: sdk.Models.Feed | sdk.Models.Channel }
type ChoiceValue = { type: string; value?: Feed | Channel } type Choice = { name: string; short?: string; value: ChoiceValue; default?: boolean }
type Choice = { name: string; short?: string; value: ChoiceValue; default?: boolean }
if (process.platform === 'win32') {
if (process.platform === 'win32') { readline
readline .createInterface({
.createInterface({ input: process.stdin,
input: process.stdin, output: process.stdout
output: process.stdout })
}) .on('SIGINT', function () {
.on('SIGINT', function () { process.emit('SIGINT')
process.emit('SIGINT') })
}) }
}
const program = new Command()
const program = new Command()
program.argument('<filepath>', 'Path to *.channels.xml file to edit').parse(process.argv)
program.argument('<filepath>', 'Path to *.channels.xml file to edit').parse(process.argv)
const filepath = program.args[0]
const filepath = program.args[0] const logger = new Logger()
const logger = new Logger() const storage = new Storage()
const storage = new Storage() let parsedStreams = new Collection<Stream>()
let parsedStreams = new Collection()
main(filepath)
main(filepath) nodeCleanup(() => {
nodeCleanup(() => { save(filepath)
save(filepath) })
})
export default async function main(filepath: string) {
export default async function main(filepath: string) { if (!(await storage.exists(filepath))) {
if (!(await storage.exists(filepath))) { throw new Error(`File "${filepath}" does not exists`)
throw new Error(`File "${filepath}" does not exists`) }
}
logger.info('loading data from api...')
logger.info('loading data from api...') await loadData()
const processor = new DataProcessor()
const dataStorage = new Storage(DATA_DIR) logger.info('loading streams...')
const loader = new DataLoader({ storage: dataStorage }) const parser = new PlaylistParser({
const data: DataLoaderData = await loader.load() storage
const { })
channels, parsedStreams = await parser.parseFile(filepath)
channelsKeyById, const streamsWithoutId = parsedStreams.filter((stream: Stream) => !stream.tvgId)
feedsGroupedByChannelId,
logosGroupedByStreamId logger.info(
}: DataProcessorData = processor.process(data) `found ${parsedStreams.count()} streams (including ${streamsWithoutId.count()} without ID)`
)
logger.info('loading streams...')
const parser = new PlaylistParser({ logger.info('starting...\n')
storage,
feedsGroupedByChannelId, for (const stream of streamsWithoutId.all()) {
logosGroupedByStreamId, try {
channelsKeyById stream.tvgId = await selectChannel(stream)
}) } catch (err) {
parsedStreams = await parser.parseFile(filepath) logger.info(err.message)
const streamsWithoutId = parsedStreams.filter((stream: Stream) => !stream.id) break
}
logger.info( }
`found ${parsedStreams.count()} streams (including ${streamsWithoutId.count()} without ID)`
) streamsWithoutId.forEach((stream: Stream) => {
if (stream.channel === '-') {
logger.info('creating search index...') stream.channel = ''
const items = channels.map((channel: Channel) => channel.getSearchable()).all() }
const searchIndex = sjs.createIndex(items, { })
searchable: ['name', 'altNames', 'guideNames', 'streamTitles', 'feedFullNames'] }
})
async function selectChannel(stream: Stream): Promise<string> {
logger.info('starting...\n') const query = escapeRegex(stream.title)
const similarChannels = searchChannels(query)
for (const stream of streamsWithoutId.all()) { const url = truncate(stream.url, 50)
try {
stream.id = await selectChannel(stream, searchIndex, feedsGroupedByChannelId, channelsKeyById) const selected: ChoiceValue = await select({
} catch (err) { message: `Select channel ID for "${stream.title}" (${url}):`,
logger.info(err.message) choices: getChannelChoises(similarChannels),
break pageSize: 10
} })
}
switch (selected.type) {
streamsWithoutId.forEach((stream: Stream) => { case 'skip':
if (stream.id === '-') { return '-'
stream.id = '' case 'type': {
} const typedChannelId = await input({ message: ' Channel ID:' })
}) if (!typedChannelId) return ''
} const selectedFeedId = await selectFeed(typedChannelId)
if (selectedFeedId === '-') return typedChannelId
async function selectChannel( return [typedChannelId, selectedFeedId].join('@')
stream: Stream, }
searchIndex, case 'channel': {
feedsGroupedByChannelId: Dictionary, const selectedChannel = selected.value
channelsKeyById: Dictionary if (!selectedChannel) return ''
): Promise<string> { const selectedFeedId = await selectFeed(selectedChannel.id)
const query = escapeRegex(stream.getTitle()) if (selectedFeedId === '-') return selectedChannel.id
const similarChannels = searchIndex return [selectedChannel.id, selectedFeedId].join('@')
.search(query) }
.map((item: ChannelSearchableData) => channelsKeyById.get(item.id)) }
const url = stream.url.length > 50 ? stream.url.slice(0, 50) + '...' : stream.url return ''
}
const selected: ChoiceValue = await select({
message: `Select channel ID for "${stream.title}" (${url}):`, async function selectFeed(channelId: string): Promise<string> {
choices: getChannelChoises(new Collection(similarChannels)), const channelFeeds = new Collection(data.feedsGroupedByChannel.get(channelId))
pageSize: 10 const choices = getFeedChoises(channelFeeds)
})
const selected: ChoiceValue = await select({
switch (selected.type) { message: `Select feed ID for "${channelId}":`,
case 'skip': choices,
return '-' pageSize: 10
case 'type': { })
const typedChannelId = await input({ message: ' Channel ID:' })
if (!typedChannelId) return '' switch (selected.type) {
const selectedFeedId = await selectFeed(typedChannelId, feedsGroupedByChannelId) case 'skip':
if (selectedFeedId === '-') return typedChannelId return '-'
return [typedChannelId, selectedFeedId].join('@') case 'type':
} return await input({ message: ' Feed ID:', default: 'SD' })
case 'channel': { case 'feed':
const selectedChannel = selected.value const selectedFeed = selected.value
if (!selectedChannel) return '' if (!selectedFeed) return ''
const selectedFeedId = await selectFeed(selectedChannel.id, feedsGroupedByChannelId) return selectedFeed.id
if (selectedFeedId === '-') return selectedChannel.id }
return [selectedChannel.id, selectedFeedId].join('@')
} return ''
} }
return '' function getChannelChoises(channels: Collection<sdk.Models.Channel>): Choice[] {
} const choises: Choice[] = []
async function selectFeed(channelId: string, feedsGroupedByChannelId: Dictionary): Promise<string> { channels.forEach((channel: sdk.Models.Channel) => {
const channelFeeds = new Collection(feedsGroupedByChannelId.get(channelId)) const names = new Collection([channel.name, ...channel.alt_names]).uniq().join(', ')
const choices = getFeedChoises(channelFeeds)
choises.push({
const selected: ChoiceValue = await select({ value: {
message: `Select feed ID for "${channelId}":`, type: 'channel',
choices, value: channel
pageSize: 10 },
}) name: `${channel.id} (${names})`,
short: `${channel.id}`
switch (selected.type) { })
case 'skip': })
return '-'
case 'type': choises.push({ name: 'Type...', value: { type: 'type' } })
return await input({ message: ' Feed ID:', default: 'SD' }) choises.push({ name: 'Skip', value: { type: 'skip' } })
case 'feed':
const selectedFeed = selected.value return choises
if (!selectedFeed) return '' }
return selectedFeed.id
} function getFeedChoises(feeds: Collection<sdk.Models.Feed>): Choice[] {
const choises: Choice[] = []
return ''
} feeds.forEach((feed: sdk.Models.Feed) => {
let name = `${feed.id} (${feed.name})`
function getChannelChoises(channels: Collection): Choice[] { if (feed.is_main) name += ' [main]'
const choises: Choice[] = []
choises.push({
channels.forEach((channel: Channel) => { value: {
const names = new Collection([channel.name, ...channel.altNames.all()]).uniq().join(', ') type: 'feed',
value: feed
choises.push({ },
value: { default: feed.is_main,
type: 'channel', name,
value: channel short: feed.id
}, })
name: `${channel.id} (${names})`, })
short: `${channel.id}`
}) choises.push({ name: 'Type...', value: { type: 'type' } })
}) choises.push({ name: 'Skip', value: { type: 'skip' } })
choises.push({ name: 'Type...', value: { type: 'type' } }) return choises
choises.push({ name: 'Skip', value: { type: 'skip' } }) }
return choises function save(filepath: string) {
} if (!storage.existsSync(filepath)) return
const playlist = new Playlist(parsedStreams)
function getFeedChoises(feeds: Collection): Choice[] { storage.saveSync(filepath, playlist.toString())
const choises: Choice[] = [] logger.info(`\nFile '${filepath}' successfully saved`)
}
feeds.forEach((feed: Feed) => {
let name = `${feed.id} (${feed.name})` function escapeRegex(string: string) {
if (feed.isMain) name += ' [main]' return string.replace(/[/\-\\^$*+?.()|[\]{}]/g, '\\$&')
}
choises.push({
value: {
type: 'feed',
value: feed
},
default: feed.isMain,
name,
short: feed.id
})
})
choises.push({ name: 'Type...', value: { type: 'type' } })
choises.push({ name: 'Skip', value: { type: 'skip' } })
return choises
}
function save(filepath: string) {
if (!storage.existsSync(filepath)) return
const playlist = new Playlist(parsedStreams)
storage.saveSync(filepath, playlist.toString())
logger.info(`\nFile '${filepath}' successfully saved`)
}
function escapeRegex(string: string) {
return string.replace(/[/\-\\^$*+?.()|[\]{}]/g, '\\$&')
}

162
scripts/commands/playlist/format.ts
View File

@@ -1,78 +1,84 @@
import { Logger, Storage } from '@freearhey/core' import { Collection, Logger } from '@freearhey/core'
import { STREAMS_DIR, DATA_DIR } from '../../constants' import { Stream, Playlist } from '../../models'
import { DataLoader, DataProcessor, PlaylistParser } from '../../core' import { Storage } from '@freearhey/storage-js'
import { Stream, Playlist } from '../../models' import { STREAMS_DIR } from '../../constants'
import { program } from 'commander' import { PlaylistParser } from '../../core'
import { DataLoaderData } from '../../types/dataLoader' import { loadData } from '../../api'
import { DataProcessorData } from '../../types/dataProcessor' import { program } from 'commander'
import path from 'node:path' import path from 'node:path'
program.argument('[filepath...]', 'Path to file to format').parse(process.argv) program.argument('[filepath...]', 'Path to file to format').parse(process.argv)
async function main() { async function main() {
const logger = new Logger() const logger = new Logger()
logger.info('loading data from api...') logger.info('loading data from api...')
const processor = new DataProcessor() await loadData()
const dataStorage = new Storage(DATA_DIR)
const loader = new DataLoader({ storage: dataStorage }) logger.info('loading streams...')
const data: DataLoaderData = await loader.load() const streamsStorage = new Storage(STREAMS_DIR)
const { channelsKeyById, feedsGroupedByChannelId, logosGroupedByStreamId }: DataProcessorData = const parser = new PlaylistParser({
processor.process(data) storage: streamsStorage
})
logger.info('loading streams...') let files = program.args.length ? program.args : await streamsStorage.list('**/*.m3u')
const streamsStorage = new Storage(STREAMS_DIR) files = files.map((filepath: string) => path.basename(filepath))
const parser = new PlaylistParser({ let streams = await parser.parse(files)
storage: streamsStorage,
channelsKeyById, logger.info(`found ${streams.count()} streams`)
feedsGroupedByChannelId,
logosGroupedByStreamId logger.info('normalizing links...')
}) streams = streams.map(stream => {
let files = program.args.length ? program.args : await streamsStorage.list('**/*.m3u') stream.normalizeURL()
files = files.map((filepath: string) => path.basename(filepath)) return stream
let streams = await parser.parse(files) })
logger.info(`found ${streams.count()} streams`) logger.info('removing duplicates...')
streams = streams.uniqBy(stream => stream.url)
logger.info('normalizing links...')
streams = streams.map(stream => { logger.info('removing wrong id...')
stream.normalizeURL() streams = streams.map((stream: Stream) => {
return stream const channel = stream.getChannel()
}) if (channel) return stream
logger.info('removing duplicates...') stream.tvgId = ''
streams = streams.uniqBy(stream => stream.url) stream.channel = ''
stream.feed = ''
logger.info('removing wrong id...')
streams = streams.map((stream: Stream) => { return stream
if (!stream.channel || channelsKeyById.missing(stream.channel.id)) { })
stream.id = ''
} logger.info('adding the missing feed id...')
streams = streams.map((stream: Stream) => {
return stream const feed = stream.getFeed()
}) if (feed) {
stream.feed = feed.id
logger.info('sorting links...') stream.tvgId = stream.getId()
streams = streams.orderBy( }
[
(stream: Stream) => stream.title, return stream
(stream: Stream) => stream.getVerticalResolution(), })
(stream: Stream) => stream.getLabel(),
(stream: Stream) => stream.url logger.info('sorting links...')
], streams = streams.sortBy(
['asc', 'desc', 'asc', 'asc'] [
) (stream: Stream) => stream.title,
(stream: Stream) => stream.getVerticalResolution(),
logger.info('saving...') (stream: Stream) => stream.label,
const groupedStreams = streams.groupBy((stream: Stream) => stream.getFilepath()) (stream: Stream) => stream.url
for (const filepath of groupedStreams.keys()) { ],
const streams = groupedStreams.get(filepath) || [] ['asc', 'desc', 'asc', 'asc']
)
if (!streams.length) return
logger.info('saving...')
const playlist = new Playlist(streams, { public: false }) const groupedStreams = streams.groupBy((stream: Stream) => stream.getFilepath())
await streamsStorage.save(filepath, playlist.toString()) for (const filepath of groupedStreams.keys()) {
} const streams = new Collection(groupedStreams.get(filepath))
}
if (streams.isEmpty()) return
main()
const playlist = new Playlist(streams, { public: false })
await streamsStorage.save(filepath, playlist.toString())
}
}
main()

246
scripts/commands/playlist/generate.ts
View File

@@ -1,131 +1,115 @@
import { PlaylistParser, DataProcessor, DataLoader } from '../../core' import { LOGS_DIR, STREAMS_DIR } from '../../constants'
import type { DataProcessorData } from '../../types/dataProcessor' import { Storage, File } from '@freearhey/storage-js'
import { DATA_DIR, LOGS_DIR, STREAMS_DIR } from '../../constants' import { PlaylistParser } from '../../core'
import type { DataLoaderData } from '../../types/dataLoader' import { loadData, data } from '../../api'
import { Logger, Storage, File } from '@freearhey/core' import { Logger } from '@freearhey/core'
import { Stream } from '../../models' import uniqueId from 'lodash.uniqueid'
import uniqueId from 'lodash.uniqueid' import { Stream } from '../../models'
import { import {
IndexCategoryGenerator, IndexCategoryGenerator,
IndexLanguageGenerator, IndexLanguageGenerator,
IndexCountryGenerator, IndexCountryGenerator,
SubdivisionsGenerator, SubdivisionsGenerator,
CategoriesGenerator, CategoriesGenerator,
CountriesGenerator, CountriesGenerator,
LanguagesGenerator, LanguagesGenerator,
RegionsGenerator, RegionsGenerator,
SourcesGenerator, SourcesGenerator,
CitiesGenerator, CitiesGenerator,
IndexGenerator, IndexGenerator,
RawGenerator RawGenerator
} from '../../generators' } from '../../generators'
async function main() { async function main() {
const logger = new Logger() const logger = new Logger()
const logFile = new File('generators.log') const logFile = new File('generators.log')
logger.info('loading data from api...') logger.info('loading data from api...')
const processor = new DataProcessor() await loadData()
const dataStorage = new Storage(DATA_DIR)
const loader = new DataLoader({ storage: dataStorage }) logger.info('loading streams...')
const data: DataLoaderData = await loader.load() const streamsStorage = new Storage(STREAMS_DIR)
const { const parser = new PlaylistParser({
feedsGroupedByChannelId, storage: streamsStorage
logosGroupedByStreamId, })
channelsKeyById, const files = await streamsStorage.list('**/*.m3u')
subdivisions, let streams = await parser.parse(files)
categories, const totalStreams = streams.count()
countries, logger.info(`found ${totalStreams} streams`)
regions,
cities logger.info('generating raw/...')
}: DataProcessorData = processor.process(data) await new RawGenerator({ streams, logFile }).generate()
logger.info('loading streams...') logger.info('filtering streams...')
const streamsStorage = new Storage(STREAMS_DIR) streams = streams.uniqBy((stream: Stream) => stream.getId() || uniqueId())
const parser = new PlaylistParser({
storage: streamsStorage, logger.info('sorting streams...')
feedsGroupedByChannelId, streams = streams.sortBy(
logosGroupedByStreamId, [
channelsKeyById (stream: Stream) => stream.getId(),
}) (stream: Stream) => stream.getVerticalResolution(),
const files = await streamsStorage.list('**/*.m3u') (stream: Stream) => stream.label
let streams = await parser.parse(files) ],
const totalStreams = streams.count() ['asc', 'asc', 'desc']
logger.info(`found ${totalStreams} streams`) )
logger.info('generating raw/...') const { categories, countries, subdivisions, cities, regions } = data
await new RawGenerator({ streams, logFile }).generate()
logger.info('generating categories/...')
logger.info('filtering streams...') await new CategoriesGenerator({ categories, streams, logFile }).generate()
streams = streams.uniqBy((stream: Stream) =>
stream.hasId() ? stream.getChannelId() + stream.getFeedId() : uniqueId() logger.info('generating languages/...')
) await new LanguagesGenerator({ streams, logFile }).generate()
logger.info('sorting streams...') logger.info('generating countries/...')
streams = streams.orderBy( await new CountriesGenerator({
[ countries,
(stream: Stream) => stream.getId(), streams,
(stream: Stream) => stream.getVerticalResolution(), logFile
(stream: Stream) => stream.getLabel() }).generate()
],
['asc', 'asc', 'desc'] logger.info('generating subdivisions/...')
) await new SubdivisionsGenerator({
subdivisions,
logger.info('generating categories/...') streams,
await new CategoriesGenerator({ categories, streams, logFile }).generate() logFile
}).generate()
logger.info('generating languages/...')
await new LanguagesGenerator({ streams, logFile }).generate() logger.info('generating cities/...')
await new CitiesGenerator({
logger.info('generating countries/...') cities,
await new CountriesGenerator({ streams,
countries, logFile
streams, }).generate()
logFile
}).generate() logger.info('generating regions/...')
await new RegionsGenerator({
logger.info('generating subdivisions/...') streams,
await new SubdivisionsGenerator({ regions,
subdivisions, logFile
streams, }).generate()
logFile
}).generate() logger.info('generating sources/...')
await new SourcesGenerator({ streams, logFile }).generate()
logger.info('generating cities/...')
await new CitiesGenerator({ logger.info('generating index.m3u...')
cities, await new IndexGenerator({ streams, logFile }).generate()
streams,
logFile logger.info('generating index.category.m3u...')
}).generate() await new IndexCategoryGenerator({ streams, logFile }).generate()
logger.info('generating regions/...') logger.info('generating index.country.m3u...')
await new RegionsGenerator({ await new IndexCountryGenerator({
streams, streams,
regions, logFile
logFile }).generate()
}).generate()
logger.info('generating index.language.m3u...')
logger.info('generating sources/...') await new IndexLanguageGenerator({ streams, logFile }).generate()
await new SourcesGenerator({ streams, logFile }).generate()
logger.info('saving generators.log...')
logger.info('generating index.m3u...') const logStorage = new Storage(LOGS_DIR)
await new IndexGenerator({ streams, logFile }).generate() logStorage.saveFile(logFile)
}
logger.info('generating index.category.m3u...')
await new IndexCategoryGenerator({ streams, logFile }).generate() main()
logger.info('generating index.country.m3u...')
await new IndexCountryGenerator({
streams,
logFile
}).generate()
logger.info('generating index.language.m3u...')
await new IndexLanguageGenerator({ streams, logFile }).generate()
logger.info('saving generators.log...')
const logStorage = new Storage(LOGS_DIR)
logStorage.saveFile(logFile)
}
main()

359
scripts/commands/playlist/test.ts
View File

@@ -1,182 +1,177 @@
import { Logger, Storage, Collection } from '@freearhey/core' import { PlaylistParser, StreamTester, CliTable } from '../../core'
import { ROOT_DIR, STREAMS_DIR, DATA_DIR } from '../../constants' import type { TestResult } from '../../core/streamTester'
import { PlaylistParser, StreamTester, CliTable, DataProcessor, DataLoader } from '../../core' import { ROOT_DIR, STREAMS_DIR } from '../../constants'
import type { TestResult } from '../../core/streamTester' import { Logger, Collection } from '@freearhey/core'
import { Stream } from '../../models' import { program, OptionValues } from 'commander'
import { program, OptionValues } from 'commander' import { Storage } from '@freearhey/storage-js'
import { eachLimit } from 'async-es' import { Stream } from '../../models'
import chalk from 'chalk' import { loadData } from '../../api'
import os from 'node:os' import { eachLimit } from 'async'
import dns from 'node:dns' import dns from 'node:dns'
import type { DataLoaderData } from '../../types/dataLoader' import chalk from 'chalk'
import type { DataProcessorData } from '../../types/dataProcessor' import os from 'node:os'
import { truncate } from '../../utils'
const LIVE_UPDATE_INTERVAL = 5000
const LIVE_UPDATE_MAX_STREAMS = 100 const LIVE_UPDATE_INTERVAL = 5000
const LIVE_UPDATE_MAX_STREAMS = 100
let errors = 0
let warnings = 0 let errors = 0
const results: { [key: string]: string } = {} let warnings = 0
let interval: string | number | NodeJS.Timeout | undefined const results: { [key: string]: string } = {}
let streams = new Collection() let interval: string | number | NodeJS.Timeout | undefined
let isLiveUpdateEnabled = true let streams = new Collection<Stream>()
let isLiveUpdateEnabled = true
program
.argument('[filepath...]', 'Path to file to test') program
.option( .argument('[filepath...]', 'Path to file to test')
'-p, --parallel <number>', .option(
'Batch size of streams to test concurrently', '-p, --parallel <number>',
(value: string) => parseInt(value), 'Batch size of streams to test concurrently',
os.cpus().length (value: string) => parseInt(value),
) os.cpus().length
.option('-x, --proxy <url>', 'Use the specified proxy') )
.option( .option('-x, --proxy <url>', 'Use the specified proxy')
'-t, --timeout <number>', .option(
'The number of milliseconds before the request will be aborted', '-t, --timeout <number>',
(value: string) => parseInt(value), 'The number of milliseconds before the request will be aborted',
30000 (value: string) => parseInt(value),
) 30000
.parse(process.argv) )
.parse(process.argv)
const options: OptionValues = program.opts()
const options: OptionValues = program.opts()
const logger = new Logger()
const tester = new StreamTester({ options }) const logger = new Logger()
const tester = new StreamTester({ options })
async function main() {
if (await isOffline()) { async function main() {
logger.error(chalk.red('Internet connection is required for the script to work')) if (await isOffline()) {
return logger.error(chalk.red('Internet connection is required for the script to work'))
} return
}
logger.info('loading data from api...')
const processor = new DataProcessor() logger.info('loading data from api...')
const dataStorage = new Storage(DATA_DIR) await loadData()
const loader = new DataLoader({ storage: dataStorage })
const data: DataLoaderData = await loader.load() logger.info('loading streams...')
const { channelsKeyById, feedsGroupedByChannelId, logosGroupedByStreamId }: DataProcessorData = const rootStorage = new Storage(ROOT_DIR)
processor.process(data) const parser = new PlaylistParser({
storage: rootStorage
logger.info('loading streams...') })
const rootStorage = new Storage(ROOT_DIR) const files = program.args.length ? program.args : await rootStorage.list(`${STREAMS_DIR}/*.m3u`)
const parser = new PlaylistParser({ streams = await parser.parse(files)
storage: rootStorage,
channelsKeyById, logger.info(`found ${streams.count()} streams`)
feedsGroupedByChannelId, if (streams.count() > LIVE_UPDATE_MAX_STREAMS) isLiveUpdateEnabled = false
logosGroupedByStreamId
}) logger.info('starting...')
const files = program.args.length ? program.args : await rootStorage.list(`${STREAMS_DIR}/*.m3u`) if (!isLiveUpdateEnabled) {
streams = await parser.parse(files) drawTable()
interval = setInterval(() => {
logger.info(`found ${streams.count()} streams`) drawTable()
if (streams.count() > LIVE_UPDATE_MAX_STREAMS) isLiveUpdateEnabled = false }, LIVE_UPDATE_INTERVAL)
}
logger.info('starting...')
if (!isLiveUpdateEnabled) { eachLimit(
drawTable() streams.all(),
interval = setInterval(() => { options.parallel,
drawTable() async (stream: Stream) => {
}, LIVE_UPDATE_INTERVAL) await runTest(stream)
}
if (isLiveUpdateEnabled) {
await eachLimit( drawTable()
streams.all(), }
options.parallel, },
async (stream: Stream) => { onFinish
await runTest(stream) )
}
if (isLiveUpdateEnabled) {
drawTable() main()
}
}, async function runTest(stream: Stream) {
onFinish const key = stream.getUniqKey()
) results[key] = chalk.white('LOADING...')
}
const result: TestResult = await tester.test(stream)
main()
let status = ''
async function runTest(stream: Stream) { const errorStatusCodes = ['ENOTFOUND', 'HTTP_404_NOT_FOUND']
const key = stream.filepath + stream.getId() + stream.url if (result.status.ok) status = chalk.green('OK')
results[key] = chalk.white('LOADING...') else if (errorStatusCodes.includes(result.status.code)) {
status = chalk.red(result.status.code)
const result: TestResult = await tester.test(stream) errors++
} else {
let status = '' status = chalk.yellow(result.status.code)
const errorStatusCodes = ['ENOTFOUND', 'HTTP_404_NOT_FOUND'] warnings++
if (result.status.ok) status = chalk.green('OK') }
else if (errorStatusCodes.includes(result.status.code)) {
status = chalk.red(result.status.code) results[key] = status
errors++ }
} else {
status = chalk.yellow(result.status.code) function drawTable() {
warnings++ process.stdout.write('\u001b[3J\u001b[1J')
} console.clear()
results[key] = status const streamsGrouped = streams.groupBy((stream: Stream) => stream.filepath)
} for (const filepath of streamsGrouped.keys()) {
const streams: Stream[] = streamsGrouped.get(filepath) || []
function drawTable() {
process.stdout.write('\u001b[3J\u001b[1J') const table = new CliTable({
console.clear() columns: [
{ name: '', alignment: 'center', minLen: 3, maxLen: 3 },
const streamsGrouped = streams.groupBy((stream: Stream) => stream.filepath) { name: 'tvg-id', alignment: 'left', color: 'green', minLen: 25, maxLen: 25 },
for (const filepath of streamsGrouped.keys()) { { name: 'url', alignment: 'left', color: 'green', minLen: 100, maxLen: 100 },
const streams: Stream[] = streamsGrouped.get(filepath) { name: 'status', alignment: 'left', minLen: 25, maxLen: 25 }
]
const table = new CliTable({ })
columns: [ streams.forEach((stream: Stream, index: number) => {
{ name: '', alignment: 'center', minLen: 3, maxLen: 3 }, const key = stream.getUniqKey()
{ name: 'tvg-id', alignment: 'left', color: 'green', minLen: 25, maxLen: 25 }, const status = results[key] || chalk.gray('PENDING')
{ name: 'url', alignment: 'left', color: 'green', minLen: 100, maxLen: 100 }, const tvgId = stream.getTvgId()
{ name: 'status', alignment: 'left', minLen: 25, maxLen: 25 }
] const row = {
}) '': index,
streams.forEach((stream: Stream, index: number) => { 'tvg-id': truncate(tvgId, 25),
const status = results[stream.filepath + stream.getId() + stream.url] || chalk.gray('PENDING') url: truncate(stream.url, 100),
status
const row = { }
'': index, table.append(row)
'tvg-id': stream.getId().length > 25 ? stream.getId().slice(0, 22) + '...' : stream.getId(), })
url: stream.url.length > 100 ? stream.url.slice(0, 97) + '...' : stream.url,
status process.stdout.write(`\n${chalk.underline(filepath)}\n`)
}
table.append(row) process.stdout.write(table.toString())
}) }
}
process.stdout.write(`\n${chalk.underline(filepath)}\n`)
function onFinish(error: Error) {
process.stdout.write(table.toString()) clearInterval(interval)
}
} if (error) {
console.error(error)
function onFinish(error: any) { process.exit(1)
clearInterval(interval) }
if (error) { drawTable()
console.error(error)
process.exit(1) if (errors > 0 || warnings > 0) {
} console.log(
chalk.red(`\n${errors + warnings} problems (${errors} errors, ${warnings} warnings)`)
drawTable() )
if (errors > 0 || warnings > 0) { if (errors > 0) {
console.log( process.exit(1)
chalk.red(`\n${errors + warnings} problems (${errors} errors, ${warnings} warnings)`) }
) }
if (errors > 0) { process.exit(0)
process.exit(1) }
}
} async function isOffline() {
return new Promise((resolve, reject) => {
process.exit(0) dns.lookup('info.cern.ch', err => {
} if (err) resolve(true)
reject(false)
async function isOffline() { })
return new Promise((resolve, reject) => { }).catch(() => {})
dns.lookup('info.cern.ch', err => { }
if (err) resolve(true)
reject(false)
})
}).catch(() => {})
}

368
scripts/commands/playlist/update.ts
View File

@@ -1,194 +1,174 @@
import { DataLoader, DataProcessor, IssueLoader, PlaylistParser } from '../../core' import { IssueLoader, PlaylistParser } from '../../core'
import { Logger, Storage, Collection, Dictionary } from '@freearhey/core' import { Playlist, Issue, Stream } from '../../models'
import type { DataProcessorData } from '../../types/dataProcessor' import { loadData, data as apiData } from '../../api'
import { Stream, Playlist, Channel, Issue } from '../../models' import { Logger, Collection } from '@freearhey/core'
import type { DataLoaderData } from '../../types/dataLoader' import { Storage } from '@freearhey/storage-js'
import { DATA_DIR, STREAMS_DIR } from '../../constants' import { STREAMS_DIR } from '../../constants'
import { isURI } from '../../utils' import * as sdk from '@iptv-org/sdk'
import { isURI } from '../../utils'
const processedIssues = new Collection()
const processedIssues = new Collection()
async function main() {
const logger = new Logger({ level: -999 }) async function main() {
const issueLoader = new IssueLoader() const logger = new Logger({ level: -999 })
const issueLoader = new IssueLoader()
logger.info('loading issues...')
const issues = await issueLoader.load() logger.info('loading issues...')
const issues = await issueLoader.load()
logger.info('loading data from api...')
const processor = new DataProcessor() logger.info('loading data from api...')
const dataStorage = new Storage(DATA_DIR) await loadData()
const dataLoader = new DataLoader({ storage: dataStorage })
const data: DataLoaderData = await dataLoader.load() logger.info('loading streams...')
const { channelsKeyById, feedsGroupedByChannelId, logosGroupedByStreamId }: DataProcessorData = const streamsStorage = new Storage(STREAMS_DIR)
processor.process(data) const parser = new PlaylistParser({
storage: streamsStorage
logger.info('loading streams...') })
const streamsStorage = new Storage(STREAMS_DIR) const files = await streamsStorage.list('**/*.m3u')
const parser = new PlaylistParser({ const streams = await parser.parse(files)
storage: streamsStorage,
feedsGroupedByChannelId, logger.info('removing streams...')
logosGroupedByStreamId, await removeStreams({ streams, issues })
channelsKeyById
}) logger.info('edit stream description...')
const files = await streamsStorage.list('**/*.m3u') await editStreams({
const streams = await parser.parse(files) streams,
issues
logger.info('removing streams...') })
await removeStreams({ streams, issues })
logger.info('add new streams...')
logger.info('edit stream description...') await addStreams({
await editStreams({ streams,
streams, issues
issues, })
channelsKeyById,
feedsGroupedByChannelId logger.info('saving...')
}) const groupedStreams = streams.groupBy((stream: Stream) => stream.getFilepath())
for (const filepath of groupedStreams.keys()) {
logger.info('add new streams...') let streams = new Collection(groupedStreams.get(filepath))
await addStreams({ streams = streams.filter((stream: Stream) => stream.removed === false)
streams,
issues, const playlist = new Playlist(streams, { public: false })
channelsKeyById, await streamsStorage.save(filepath, playlist.toString())
feedsGroupedByChannelId }
})
const output = processedIssues.map(issue_number => `closes #${issue_number}`).join(', ')
logger.info('saving...') console.log(`OUTPUT=${output}`)
const groupedStreams = streams.groupBy((stream: Stream) => stream.getFilepath()) }
for (const filepath of groupedStreams.keys()) {
let streams = groupedStreams.get(filepath) || [] main()
streams = streams.filter((stream: Stream) => stream.removed === false)
async function removeStreams({
const playlist = new Playlist(streams, { public: false }) streams,
await streamsStorage.save(filepath, playlist.toString()) issues
} }: {
streams: Collection<Stream>
const output = processedIssues.map(issue_number => `closes #${issue_number}`).join(', ') issues: Collection<Issue>
console.log(`OUTPUT=${output}`) }) {
} const requests = issues.filter(
issue => issue.labels.includes('streams:remove') && issue.labels.includes('approved')
main() )
async function removeStreams({ streams, issues }: { streams: Collection; issues: Collection }) { requests.forEach((issue: Issue) => {
const requests = issues.filter( const data = issue.data
issue => issue.labels.includes('streams:remove') && issue.labels.includes('approved') if (data.missing('streamUrl')) return
)
requests.forEach((issue: Issue) => { const streamUrls = data.getString('streamUrl') || ''
const data = issue.data
if (data.missing('streamUrl')) return let changed = false
streamUrls
const streamUrls = data.getString('streamUrl') || '' .split(/\r?\n/)
.filter(Boolean)
let changed = false .forEach(link => {
streamUrls const found: Stream = streams.first((_stream: Stream) => _stream.url === link.trim())
.split(/\r?\n/) if (found) {
.filter(Boolean) found.removed = true
.forEach(link => { changed = true
const found: Stream = streams.first((_stream: Stream) => _stream.url === link.trim()) }
if (found) { })
found.removed = true
changed = true if (changed) processedIssues.add(issue.number)
} })
}) }
if (changed) processedIssues.add(issue.number) async function editStreams({
}) streams,
} issues
}: {
async function editStreams({ streams: Collection<Stream>
streams, issues: Collection<Issue>
issues, }) {
channelsKeyById, const requests = issues.filter(
feedsGroupedByChannelId issue => issue.labels.includes('streams:edit') && issue.labels.includes('approved')
}: { )
streams: Collection requests.forEach((issue: Issue) => {
issues: Collection const data = issue.data
channelsKeyById: Dictionary
feedsGroupedByChannelId: Dictionary if (data.missing('streamUrl')) return
}) {
const requests = issues.filter( const stream: Stream = streams.first(
issue => issue.labels.includes('streams:edit') && issue.labels.includes('approved') (_stream: Stream) => _stream.url === data.getString('streamUrl')
) )
requests.forEach((issue: Issue) => { if (!stream) return
const data = issue.data
const streamId = data.getString('streamId') || ''
if (data.missing('streamUrl')) return const [channelId, feedId] = streamId.split('@')
const stream: Stream = streams.first( if (channelId) {
(_stream: Stream) => _stream.url === data.getString('streamUrl') stream.channel = channelId
) stream.feed = feedId
if (!stream) return stream.updateTvgId().updateTitle().updateFilepath()
}
const streamId = data.getString('streamId') || ''
const [channelId, feedId] = streamId.split('@') stream.updateWithIssue(data)
if (channelId) { processedIssues.add(issue.number)
stream })
.setChannelId(channelId) }
.setFeedId(feedId)
.withChannel(channelsKeyById) async function addStreams({
.withFeed(feedsGroupedByChannelId) streams,
.updateId() issues
.updateTitle() }: {
.updateFilepath() streams: Collection<Stream>
} issues: Collection<Issue>
}) {
stream.update(data) const requests = issues.filter(
issue => issue.labels.includes('streams:add') && issue.labels.includes('approved')
processedIssues.add(issue.number) )
}) requests.forEach((issue: Issue) => {
} const data = issue.data
if (data.missing('streamId') || data.missing('streamUrl')) return
async function addStreams({ if (streams.includes((_stream: Stream) => _stream.url === data.getString('streamUrl'))) return
streams, const streamUrl = data.getString('streamUrl') || ''
issues, if (!isURI(streamUrl)) return
channelsKeyById,
feedsGroupedByChannelId const streamId = data.getString('streamId') || ''
}: { const [channelId, feedId] = streamId.split('@')
streams: Collection
issues: Collection const channel: sdk.Models.Channel | undefined = apiData.channelsKeyById.get(channelId)
channelsKeyById: Dictionary if (!channel) return
feedsGroupedByChannelId: Dictionary
}) { const label = data.getString('label') || ''
const requests = issues.filter( const quality = data.getString('quality') || null
issue => issue.labels.includes('streams:add') && issue.labels.includes('approved') const httpUserAgent = data.getString('httpUserAgent') || null
) const httpReferrer = data.getString('httpReferrer') || null
requests.forEach((issue: Issue) => { const directives = data.getArray('directives') || []
const data = issue.data
if (data.missing('streamId') || data.missing('streamUrl')) return const stream = new Stream({
if (streams.includes((_stream: Stream) => _stream.url === data.getString('streamUrl'))) return channel: channelId,
const streamUrl = data.getString('streamUrl') || '' feed: feedId,
if (!isURI(streamUrl)) return title: channel.name,
url: streamUrl,
const streamId = data.getString('streamId') || '' user_agent: httpUserAgent,
const [channelId, feedId] = streamId.split('@') referrer: httpReferrer,
quality
const channel: Channel = channelsKeyById.get(channelId) })
if (!channel) return
stream.label = label
const label = data.getString('label') || null stream.setDirectives(directives).updateTitle().updateFilepath()
const quality = data.getString('quality') || null
const httpUserAgent = data.getString('httpUserAgent') || null streams.add(stream)
const httpReferrer = data.getString('httpReferrer') || null processedIssues.add(issue.number)
const directives = data.getArray('directives') || [] })
}
const stream = new Stream({
channelId,
feedId,
title: channel.name,
url: streamUrl,
userAgent: httpUserAgent,
referrer: httpReferrer,
directives,
quality,
label
})
.withChannel(channelsKeyById)
.withFeed(feedsGroupedByChannelId)
.updateTitle()
.updateFilepath()
streams.add(stream)
processedIssues.add(issue.number)
})
}

249
scripts/commands/playlist/validate.ts
View File

@@ -1,129 +1,120 @@
import { Logger, Storage, Collection, Dictionary } from '@freearhey/core' import { Logger, Collection, Dictionary } from '@freearhey/core'
import { DataLoader, DataProcessor, PlaylistParser } from '../../core' import { Storage } from '@freearhey/storage-js'
import { DataProcessorData } from '../../types/dataProcessor' import { PlaylistParser } from '../../core'
import { DATA_DIR, ROOT_DIR } from '../../constants' import { data, loadData } from '../../api'
import { DataLoaderData } from '../../types/dataLoader' import { ROOT_DIR } from '../../constants'
import { BlocklistRecord, Stream } from '../../models' import { Stream } from '../../models'
import { program } from 'commander' import * as sdk from '@iptv-org/sdk'
import chalk from 'chalk' import { program } from 'commander'
import chalk from 'chalk'
program.argument('[filepath...]', 'Path to file to validate').parse(process.argv)
program.argument('[filepath...]', 'Path to file to validate').parse(process.argv)
type LogItem = {
type: string type LogItem = {
line: number type: string
message: string line: number
} message: string
}
async function main() {
const logger = new Logger() async function main() {
const logger = new Logger()
logger.info('loading data from api...')
const processor = new DataProcessor() logger.info('loading data from api...')
const dataStorage = new Storage(DATA_DIR) await loadData()
const loader = new DataLoader({ storage: dataStorage })
const data: DataLoaderData = await loader.load() logger.info('loading streams...')
const { const rootStorage = new Storage(ROOT_DIR)
channelsKeyById, const parser = new PlaylistParser({
feedsGroupedByChannelId, storage: rootStorage
logosGroupedByStreamId, })
blocklistRecordsGroupedByChannelId const files = program.args.length ? program.args : await rootStorage.list('streams/**/*.m3u')
}: DataProcessorData = processor.process(data) const streams = await parser.parse(files)
logger.info(`found ${streams.count()} streams`)
logger.info('loading streams...')
const rootStorage = new Storage(ROOT_DIR) let errors = new Collection()
const parser = new PlaylistParser({ let warnings = new Collection()
storage: rootStorage, const streamsGroupedByFilepath = streams.groupBy((stream: Stream) => stream.getFilepath())
channelsKeyById, for (const filepath of streamsGroupedByFilepath.keys()) {
feedsGroupedByChannelId, const streams = streamsGroupedByFilepath.get(filepath)
logosGroupedByStreamId if (!streams) continue
})
const files = program.args.length ? program.args : await rootStorage.list('streams/**/*.m3u') const log = new Collection<LogItem>()
const streams = await parser.parse(files) const buffer = new Dictionary<boolean>()
logger.info(`found ${streams.count()} streams`) streams.forEach((stream: Stream) => {
if (stream.channel) {
let errors = new Collection() const channel = data.channelsKeyById.get(stream.channel)
let warnings = new Collection() if (!channel) {
const streamsGroupedByFilepath = streams.groupBy((stream: Stream) => stream.getFilepath()) log.add({
for (const filepath of streamsGroupedByFilepath.keys()) { type: 'warning',
const streams = streamsGroupedByFilepath.get(filepath) line: stream.getLine(),
if (!streams) continue message: `"${stream.tvgId}" is not in the database`
})
const log = new Collection() }
const buffer = new Dictionary() }
streams.forEach((stream: Stream) => {
if (stream.channelId) { const duplicate = stream.url && buffer.has(stream.url)
const channel = channelsKeyById.get(stream.channelId) if (duplicate) {
if (!channel) { log.add({
log.add({ type: 'warning',
type: 'warning', line: stream.getLine(),
line: stream.getLine(), message: `"${stream.url}" is already on the playlist`
message: `"${stream.id}" is not in the database` })
}) } else {
} buffer.set(stream.url, true)
} }
const duplicate = stream.url && buffer.has(stream.url) if (stream.channel) {
if (duplicate) { const blocklistRecords = new Collection(
log.add({ data.blocklistRecordsGroupedByChannel.get(stream.channel)
type: 'warning', )
line: stream.getLine(),
message: `"${stream.url}" is already on the playlist` blocklistRecords.forEach((blocklistRecord: sdk.Models.BlocklistRecord) => {
}) if (blocklistRecord.reason === 'dmca') {
} else { log.add({
buffer.set(stream.url, true) type: 'error',
} line: stream.getLine(),
message: `"${blocklistRecord.channel}" is on the blocklist due to claims of copyright holders (${blocklistRecord.ref})`
const blocklistRecords = stream.channel })
? new Collection(blocklistRecordsGroupedByChannelId.get(stream.channel.id)) } else if (blocklistRecord.reason === 'nsfw') {
: new Collection() log.add({
type: 'error',
blocklistRecords.forEach((blocklistRecord: BlocklistRecord) => { line: stream.getLine(),
if (blocklistRecord.reason === 'dmca') { message: `"${blocklistRecord.channel}" is on the blocklist due to NSFW content (${blocklistRecord.ref})`
log.add({ })
type: 'error', }
line: stream.getLine(), })
message: `"${blocklistRecord.channelId}" is on the blocklist due to claims of copyright holders (${blocklistRecord.ref})` }
}) })
} else if (blocklistRecord.reason === 'nsfw') {
log.add({ if (log.isNotEmpty()) {
type: 'error', console.log(`\n${chalk.underline(filepath)}`)
line: stream.getLine(),
message: `"${blocklistRecord.channelId}" is on the blocklist due to NSFW content (${blocklistRecord.ref})` log.forEach((logItem: LogItem) => {
}) const position = logItem.line.toString().padEnd(6, ' ')
} const type = logItem.type.padEnd(9, ' ')
}) const status = logItem.type === 'error' ? chalk.red(type) : chalk.yellow(type)
})
console.log(` ${chalk.gray(position)}${status}${logItem.message}`)
if (log.notEmpty()) { })
console.log(`\n${chalk.underline(filepath)}`)
errors = errors.concat(log.filter((logItem: LogItem) => logItem.type === 'error'))
log.forEach((logItem: LogItem) => { warnings = warnings.concat(log.filter((logItem: LogItem) => logItem.type === 'warning'))
const position = logItem.line.toString().padEnd(6, ' ') }
const type = logItem.type.padEnd(9, ' ') }
const status = logItem.type === 'error' ? chalk.red(type) : chalk.yellow(type)
if (errors.count() || warnings.count()) {
console.log(` ${chalk.gray(position)}${status}${logItem.message}`) console.log(
}) chalk.red(
`\n${
errors = errors.concat(log.filter((logItem: LogItem) => logItem.type === 'error')) errors.count() + warnings.count()
warnings = warnings.concat(log.filter((logItem: LogItem) => logItem.type === 'warning')) } problems (${errors.count()} errors, ${warnings.count()} warnings)`
} )
} )
if (errors.count() || warnings.count()) { if (errors.count()) {
console.log( process.exit(1)
chalk.red( }
`\n${ }
errors.count() + warnings.count() }
} problems (${errors.count()} errors, ${warnings.count()} warnings)`
) main()
)
if (errors.count()) {
process.exit(1)
}
}
}
main()

78
scripts/commands/readme/update.ts
View File

@@ -1,48 +1,30 @@
import { CategoriesTable, CountriesTable, LanguagesTable, RegionsTable } from '../../tables' import { CategoriesTable, CountriesTable, LanguagesTable, RegionsTable } from '../../tables'
import { DataLoader, DataProcessor, Markdown } from '../../core' import { README_DIR, ROOT_DIR } from '../../constants'
import { DataProcessorData } from '../../types/dataProcessor' import { Logger } from '@freearhey/core'
import { DataLoaderData } from '../../types/dataLoader' import { Markdown } from '../../core'
import { README_DIR, DATA_DIR, ROOT_DIR } from '../../constants' import { loadData } from '../../api'
import { Logger, Storage } from '@freearhey/core'
async function main() {
async function main() { const logger = new Logger()
const logger = new Logger()
const dataStorage = new Storage(DATA_DIR) logger.info('loading data from api...')
const processor = new DataProcessor() await loadData()
const loader = new DataLoader({ storage: dataStorage })
const data: DataLoaderData = await loader.load() logger.info('creating category table...')
const { await new CategoriesTable().create()
subdivisionsKeyByCode, logger.info('creating language table...')
languagesKeyByCode, await new LanguagesTable().create()
countriesKeyByCode, logger.info('creating countires table...')
categoriesKeyById, await new CountriesTable().create()
subdivisions, logger.info('creating region table...')
countries, await new RegionsTable().create()
regions,
cities logger.info('updating playlists.md...')
}: DataProcessorData = processor.process(data) const playlists = new Markdown({
build: `${ROOT_DIR}/PLAYLISTS.md`,
logger.info('creating category table...') template: `${README_DIR}/template.md`
await new CategoriesTable({ categoriesKeyById }).make() })
logger.info('creating language table...') playlists.compile()
await new LanguagesTable({ languagesKeyByCode }).make() }
logger.info('creating countires table...')
await new CountriesTable({ main()
countriesKeyByCode,
subdivisionsKeyByCode,
subdivisions,
countries,
cities
}).make()
logger.info('creating region table...')
await new RegionsTable({ regions }).make()
logger.info('updating playlists.md...')
const playlists = new Markdown({
build: `${ROOT_DIR}/PLAYLISTS.md`,
template: `${README_DIR}/template.md`
})
playlists.compile()
}
main()

354
scripts/commands/report/create.ts
View File

@@ -1,178 +1,176 @@
import { DataLoader, DataProcessor, IssueLoader, PlaylistParser } from '../../core' import { Logger, Collection, Dictionary } from '@freearhey/core'
import { Logger, Storage, Collection, Dictionary } from '@freearhey/core' import { IssueLoader, PlaylistParser } from '../../core'
import { DataProcessorData } from '../../types/dataProcessor' import { Storage } from '@freearhey/storage-js'
import { DATA_DIR, STREAMS_DIR } from '../../constants' import { isURI, truncate } from '../../utils'
import { DataLoaderData } from '../../types/dataLoader' import { STREAMS_DIR } from '../../constants'
import { Issue, Stream } from '../../models' import { Issue, Stream } from '../../models'
import { isURI } from '../../utils' import { data, loadData } from '../../api'
async function main() { const status = {
const logger = new Logger() PENDING: 'pending',
const issueLoader = new IssueLoader() FULFILLED: 'fulfilled',
let report = new Collection() MISSING_CHANNEL_ID: 'missing_channel_id',
INVALID_CHANNEL_ID: 'invalid_channel_id',
logger.info('loading issues...') MISSING_STREAM_URL: 'missing_stream_url',
const issues = await issueLoader.load() INVALID_STREAM_URL: 'invalid_stream_url',
NONEXISTENT_LINK: 'nonexistent_link',
logger.info('loading data from api...') CHANNEL_BLOCKED: 'channel_blocked',
const processor = new DataProcessor() CHANNEL_CLOSED: 'channel_closed',
const dataStorage = new Storage(DATA_DIR) DUPLICATE_LINK: 'duplicate_link',
const dataLoader = new DataLoader({ storage: dataStorage }) DUPLICATE_REQUEST: 'duplicate_request'
const data: DataLoaderData = await dataLoader.load() }
const {
channelsKeyById, async function main() {
feedsGroupedByChannelId, const logger = new Logger()
logosGroupedByStreamId, const issueLoader = new IssueLoader()
blocklistRecordsGroupedByChannelId let report = new Collection()
}: DataProcessorData = processor.process(data)
logger.info('loading issues...')
logger.info('loading streams...') const issues = await issueLoader.load()
const streamsStorage = new Storage(STREAMS_DIR)
const parser = new PlaylistParser({ logger.info('loading data from api...')
storage: streamsStorage, await loadData()
channelsKeyById,
feedsGroupedByChannelId, logger.info('loading streams...')
logosGroupedByStreamId const streamsStorage = new Storage(STREAMS_DIR)
}) const parser = new PlaylistParser({
const files = await streamsStorage.list('**/*.m3u') storage: streamsStorage
const streams = await parser.parse(files) })
const streamsGroupedByUrl = streams.groupBy((stream: Stream) => stream.url) const files = await streamsStorage.list('**/*.m3u')
const streamsGroupedByChannelId = streams.groupBy((stream: Stream) => stream.channelId) const streams = await parser.parse(files)
const streamsGroupedById = streams.groupBy((stream: Stream) => stream.getId()) const streamsGroupedByUrl = streams.groupBy((stream: Stream) => stream.url)
const streamsGroupedByChannel = streams.groupBy((stream: Stream) => stream.channel)
logger.info('checking streams:remove requests...') const streamsGroupedById = streams.groupBy((stream: Stream) => stream.getId())
const removeRequests = issues.filter(issue =>
issue.labels.find((label: string) => label === 'streams:remove') logger.info('checking streams:remove requests...')
) const removeRequests = issues.filter(issue =>
removeRequests.forEach((issue: Issue) => { issue.labels.find((label: string) => label === 'streams:remove')
const streamUrls = issue.data.getArray('streamUrl') || [] )
removeRequests.forEach((issue: Issue) => {
if (!streamUrls.length) { const streamUrls = issue.data.getArray('streamUrl') || []
const result = {
issueNumber: issue.number, if (!streamUrls.length) {
type: 'streams:remove', const result = {
streamId: undefined, issueNumber: issue.number,
streamUrl: undefined, type: 'streams:remove',
status: 'missing_link' streamId: undefined,
} streamUrl: undefined,
status: status.NONEXISTENT_LINK
report.add(result) }
} else {
for (const streamUrl of streamUrls) { report.add(result)
const result = { } else {
issueNumber: issue.number, for (const streamUrl of streamUrls) {
type: 'streams:remove', const result = {
streamId: undefined, issueNumber: issue.number,
streamUrl: truncate(streamUrl), type: 'streams:remove',
status: 'pending' streamId: undefined,
} streamUrl: truncate(streamUrl),
status: status.PENDING
if (streamsGroupedByUrl.missing(streamUrl)) { }
result.status = 'wrong_link'
} if (streamsGroupedByUrl.missing(streamUrl)) {
result.status = status.NONEXISTENT_LINK
report.add(result) }
}
} report.add(result)
}) }
}
logger.info('checking streams:add requests...') })
const addRequests = issues.filter(issue => issue.labels.includes('streams:add'))
const addRequestsBuffer = new Dictionary() logger.info('checking streams:add requests...')
addRequests.forEach((issue: Issue) => { const addRequests = issues.filter(issue => issue.labels.includes('streams:add'))
const streamId = issue.data.getString('streamId') || '' const addRequestsBuffer = new Dictionary()
const streamUrl = issue.data.getString('streamUrl') || '' addRequests.forEach((issue: Issue) => {
const [channelId] = streamId.split('@') const streamId = issue.data.getString('streamId') || ''
const streamUrl = issue.data.getString('streamUrl') || ''
const result = { const [channelId] = streamId.split('@')
issueNumber: issue.number,
type: 'streams:add', const result = {
streamId: streamId || undefined, issueNumber: issue.number,
streamUrl: truncate(streamUrl), type: 'streams:add',
status: 'pending' streamId: streamId || undefined,
} streamUrl: truncate(streamUrl),
status: status.PENDING
if (!channelId) result.status = 'missing_id' }
else if (!streamUrl) result.status = 'missing_link'
else if (!isURI(streamUrl)) result.status = 'invalid_link' if (!channelId) result.status = status.MISSING_CHANNEL_ID
else if (blocklistRecordsGroupedByChannelId.has(channelId)) result.status = 'blocked' else if (!streamUrl) result.status = status.MISSING_STREAM_URL
else if (channelsKeyById.missing(channelId)) result.status = 'wrong_id' else if (!isURI(streamUrl)) result.status = status.INVALID_STREAM_URL
else if (streamsGroupedByUrl.has(streamUrl)) result.status = 'on_playlist' else if (data.blocklistRecordsGroupedByChannel.has(channelId))
else if (addRequestsBuffer.has(streamUrl)) result.status = 'duplicate' result.status = status.CHANNEL_BLOCKED
else result.status = 'pending' else if (data.channelsKeyById.missing(channelId)) result.status = status.INVALID_CHANNEL_ID
else if (streamsGroupedByUrl.has(streamUrl)) result.status = status.DUPLICATE_LINK
addRequestsBuffer.set(streamUrl, true) else if (addRequestsBuffer.has(streamUrl)) result.status = status.DUPLICATE_REQUEST
else result.status = status.PENDING
report.add(result)
}) addRequestsBuffer.set(streamUrl, true)
logger.info('checking streams:edit requests...') report.add(result)
const editRequests = issues.filter(issue => })
issue.labels.find((label: string) => label === 'streams:edit')
) logger.info('checking streams:edit requests...')
editRequests.forEach((issue: Issue) => { const editRequests = issues.filter(issue =>
const streamId = issue.data.getString('streamId') || '' issue.labels.find((label: string) => label === 'streams:edit')
const streamUrl = issue.data.getString('streamUrl') || '' )
const [channelId] = streamId.split('@') editRequests.forEach((issue: Issue) => {
const streamId = issue.data.getString('streamId') || ''
const result = { const streamUrl = issue.data.getString('streamUrl') || ''
issueNumber: issue.number, const [channelId] = streamId.split('@')
type: 'streams:edit',
streamId: streamId || undefined, const result = {
streamUrl: truncate(streamUrl), issueNumber: issue.number,
status: 'pending' type: 'streams:edit',
} streamId: streamId || undefined,
streamUrl: truncate(streamUrl),
if (!streamUrl) result.status = 'missing_link' status: status.PENDING
else if (streamsGroupedByUrl.missing(streamUrl)) result.status = 'invalid_link' }
else if (channelId && channelsKeyById.missing(channelId)) result.status = 'invalid_id'
if (!streamUrl) result.status = status.MISSING_STREAM_URL
report.add(result) else if (streamsGroupedByUrl.missing(streamUrl)) result.status = status.NONEXISTENT_LINK
}) else if (channelId && data.channelsKeyById.missing(channelId))
result.status = status.INVALID_CHANNEL_ID
logger.info('checking channel search requests...')
const channelSearchRequests = issues.filter(issue => report.add(result)
issue.labels.find((label: string) => label === 'channel search') })
)
const channelSearchRequestsBuffer = new Dictionary() logger.info('checking channel search requests...')
channelSearchRequests.forEach((issue: Issue) => { const channelSearchRequests = issues.filter(issue =>
const streamId = issue.data.getString('channelId') || '' issue.labels.find((label: string) => label === 'channel search')
const [channelId, feedId] = streamId.split('@') )
const channelSearchRequestsBuffer = new Dictionary()
const result = { channelSearchRequests.forEach((issue: Issue) => {
issueNumber: issue.number, const streamId = issue.data.getString('streamId') || issue.data.getString('channelId') || ''
type: 'channel search', const [channelId, feedId] = streamId.split('@')
streamId: streamId || undefined,
streamUrl: undefined, const result = {
status: 'pending' issueNumber: issue.number,
} type: 'channel search',
streamId: streamId || undefined,
if (!channelId) result.status = 'missing_id' streamUrl: undefined,
else if (channelsKeyById.missing(channelId)) result.status = 'invalid_id' status: status.PENDING
else if (channelSearchRequestsBuffer.has(streamId)) result.status = 'duplicate' }
else if (blocklistRecordsGroupedByChannelId.has(channelId)) result.status = 'blocked'
else if (streamsGroupedById.has(streamId)) result.status = 'fulfilled' if (!channelId) result.status = status.MISSING_CHANNEL_ID
else if (!feedId && streamsGroupedByChannelId.has(channelId)) result.status = 'fulfilled' else if (data.channelsKeyById.missing(channelId)) result.status = status.INVALID_CHANNEL_ID
else { else if (channelSearchRequestsBuffer.has(streamId)) result.status = status.DUPLICATE_REQUEST
const channelData = channelsKeyById.get(channelId) else if (data.blocklistRecordsGroupedByChannel.has(channelId))
if (channelData && channelData.isClosed) result.status = 'closed' result.status = status.CHANNEL_BLOCKED
} else if (streamsGroupedById.has(streamId)) result.status = status.FULFILLED
else if (!feedId && streamsGroupedByChannel.has(channelId)) result.status = status.FULFILLED
channelSearchRequestsBuffer.set(streamId, true) else {
const channelData = data.channelsKeyById.get(channelId)
report.add(result) if (channelData && channelData.isClosed()) result.status = status.CHANNEL_CLOSED
}) }
report = report.orderBy(item => item.issueNumber).filter(item => item.status !== 'pending') channelSearchRequestsBuffer.set(streamId, true)
console.table(report.all()) report.add(result)
} })
main() report = report.sortBy(item => item.issueNumber).filter(item => item.status !== status.PENDING)
function truncate(string: string, limit: number = 100) { console.table(report.all())
if (!string) return string }
if (string.length < limit) return string
main()
return string.slice(0, limit) + '...'
}

22
scripts/constants.ts
View File

@@ -1,11 +1,11 @@
export const ROOT_DIR = process.env.ROOT_DIR || './' export const ROOT_DIR = process.env.ROOT_DIR || './'
export const STREAMS_DIR = process.env.STREAMS_DIR || './streams' export const STREAMS_DIR = process.env.STREAMS_DIR || './streams'
export const PUBLIC_DIR = process.env.PUBLIC_DIR || './.gh-pages' export const PUBLIC_DIR = process.env.PUBLIC_DIR || './.gh-pages'
export const README_DIR = process.env.README_DIR || './.readme' export const README_DIR = process.env.README_DIR || './.readme'
export const API_DIR = process.env.API_DIR || './.api' export const API_DIR = process.env.API_DIR || './.api'
export const DATA_DIR = process.env.DATA_DIR || './temp/data' export const DATA_DIR = process.env.DATA_DIR || './temp/data'
export const LOGS_DIR = process.env.LOGS_DIR || './temp/logs' export const LOGS_DIR = process.env.LOGS_DIR || './temp/logs'
export const TESTING = process.env.NODE_ENV === 'test' ? true : false export const TESTING = process.env.NODE_ENV === 'test' ? true : false
export const OWNER = 'iptv-org' export const OWNER = 'iptv-org'
export const REPO = 'iptv' export const REPO = 'iptv'
export const EOL = '\r\n' export const EOL = '\r\n'

16
scripts/core/apiClient.ts
View File

@@ -1,16 +0,0 @@
import axios, { AxiosInstance, AxiosResponse, AxiosRequestConfig } from 'axios'
export class ApiClient {
instance: AxiosInstance
constructor() {
this.instance = axios.create({
baseURL: 'https://iptv-org.github.io/api',
responseType: 'stream'
})
}
get(url: string, options: AxiosRequestConfig): Promise<AxiosResponse> {
return this.instance.get(url, options)
}
}

44
scripts/core/cliTable.ts
View File

@@ -1,22 +1,22 @@
import { Table } from 'console-table-printer' import { ComplexOptions } from 'console-table-printer/dist/src/models/external-table'
import { ComplexOptions } from 'console-table-printer/dist/src/models/external-table' import { Table } from 'console-table-printer'
export class CliTable { export class CliTable {
table: Table table: Table
constructor(options?: ComplexOptions | string[]) { constructor(options?: ComplexOptions | string[]) {
this.table = new Table(options) this.table = new Table(options)
} }
append(row) { append(row) {
this.table.addRow(row) this.table.addRow(row)
} }
render() { render() {
this.table.printTable() this.table.printTable()
} }
toString() { toString() {
return this.table.render() return this.table.render()
} }
} }

113
scripts/core/dataLoader.ts
View File

@@ -1,113 +0,0 @@
import { ApiClient } from './apiClient'
import { Storage } from '@freearhey/core'
import cliProgress, { MultiBar } from 'cli-progress'
import type { DataLoaderProps, DataLoaderData } from '../types/dataLoader'
const formatBytes = (bytes: number) => {
if (bytes === 0) return '0 B'
const k = 1024
const sizes = ['B', 'KB', 'MB', 'GB']
const i = Math.floor(Math.log(bytes) / Math.log(k))
return parseFloat((bytes / Math.pow(k, i)).toFixed(1)) + ' ' + sizes[i]
}
export class DataLoader {
client: ApiClient
storage: Storage
progressBar: MultiBar
constructor(props: DataLoaderProps) {
this.client = new ApiClient()
this.storage = props.storage
this.progressBar = new cliProgress.MultiBar({
stopOnComplete: true,
hideCursor: true,
forceRedraw: true,
barsize: 36,
format(options, params, payload) {
const filename = payload.filename.padEnd(18, ' ')
const barsize = options.barsize || 40
const percent = (params.progress * 100).toFixed(2)
const speed = payload.speed ? formatBytes(payload.speed) + '/s' : 'N/A'
const total = formatBytes(params.total)
const completeSize = Math.round(params.progress * barsize)
const incompleteSize = barsize - completeSize
const bar =
options.barCompleteString && options.barIncompleteString
? options.barCompleteString.substr(0, completeSize) +
options.barGlue +
options.barIncompleteString.substr(0, incompleteSize)
: '-'.repeat(barsize)
return `${filename} [${bar}] ${percent}% | ETA: ${params.eta}s | ${total} | ${speed}`
}
})
}
async load(): Promise<DataLoaderData> {
const [
countries,
regions,
subdivisions,
languages,
categories,
blocklist,
channels,
feeds,
logos,
timezones,
guides,
streams,
cities
] = await Promise.all([
this.storage.json('countries.json'),
this.storage.json('regions.json'),
this.storage.json('subdivisions.json'),
this.storage.json('languages.json'),
this.storage.json('categories.json'),
this.storage.json('blocklist.json'),
this.storage.json('channels.json'),
this.storage.json('feeds.json'),
this.storage.json('logos.json'),
this.storage.json('timezones.json'),
this.storage.json('guides.json'),
this.storage.json('streams.json'),
this.storage.json('cities.json')
])
return {
countries,
regions,
subdivisions,
languages,
categories,
blocklist,
channels,
feeds,
logos,
timezones,
guides,
streams,
cities
}
}
async download(filename: string) {
if (!this.storage || !this.progressBar) return
const stream = await this.storage.createStream(filename)
const progressBar = this.progressBar.create(0, 0, { filename })
this.client
.get(filename, {
responseType: 'stream',
onDownloadProgress({ total, loaded, rate }) {
if (total) progressBar.setTotal(total)
progressBar.update(loaded, { speed: rate })
}
})
.then(response => {
response.data.pipe(stream)
})
}
}

165
scripts/core/dataProcessor.ts
View File

@@ -1,165 +0,0 @@
import { DataProcessorData } from '../types/dataProcessor'
import { DataLoaderData } from '../types/dataLoader'
import { Collection } from '@freearhey/core'
import {
BlocklistRecord,
Subdivision,
Category,
Language,
Timezone,
Channel,
Country,
Region,
Stream,
Guide,
City,
Feed,
Logo
} from '../models'
export class DataProcessor {
process(data: DataLoaderData): DataProcessorData {
let regions = new Collection(data.regions).map(data => new Region(data))
let regionsKeyByCode = regions.keyBy((region: Region) => region.code)
const categories = new Collection(data.categories).map(data => new Category(data))
const categoriesKeyById = categories.keyBy((category: Category) => category.id)
const languages = new Collection(data.languages).map(data => new Language(data))
const languagesKeyByCode = languages.keyBy((language: Language) => language.code)
let subdivisions = new Collection(data.subdivisions).map(data => new Subdivision(data))
let subdivisionsKeyByCode = subdivisions.keyBy((subdivision: Subdivision) => subdivision.code)
let subdivisionsGroupedByCountryCode = subdivisions.groupBy(
(subdivision: Subdivision) => subdivision.countryCode
)
let countries = new Collection(data.countries).map(data => new Country(data))
let countriesKeyByCode = countries.keyBy((country: Country) => country.code)
const cities = new Collection(data.cities).map(data =>
new City(data)
.withRegions(regions)
.withCountry(countriesKeyByCode)
.withSubdivision(subdivisionsKeyByCode)
)
const citiesKeyByCode = cities.keyBy((city: City) => city.code)
const citiesGroupedByCountryCode = cities.groupBy((city: City) => city.countryCode)
const citiesGroupedBySubdivisionCode = cities.groupBy((city: City) => city.subdivisionCode)
const timezones = new Collection(data.timezones).map(data =>
new Timezone(data).withCountries(countriesKeyByCode)
)
const timezonesKeyById = timezones.keyBy((timezone: Timezone) => timezone.id)
const blocklistRecords = new Collection(data.blocklist).map(data => new BlocklistRecord(data))
const blocklistRecordsGroupedByChannelId = blocklistRecords.groupBy(
(blocklistRecord: BlocklistRecord) => blocklistRecord.channelId
)
let channels = new Collection(data.channels).map(data => new Channel(data))
let channelsKeyById = channels.keyBy((channel: Channel) => channel.id)
let feeds = new Collection(data.feeds).map(data => new Feed(data))
let feedsGroupedByChannelId = feeds.groupBy((feed: Feed) => feed.channelId)
let feedsGroupedById = feeds.groupBy((feed: Feed) => feed.id)
const logos = new Collection(data.logos).map(data => new Logo(data).withFeed(feedsGroupedById))
const logosGroupedByChannelId = logos.groupBy((logo: Logo) => logo.channelId)
const logosGroupedByStreamId = logos.groupBy((logo: Logo) => logo.getStreamId())
const streams = new Collection(data.streams).map(data =>
new Stream(data).withLogos(logosGroupedByStreamId)
)
const streamsGroupedById = streams.groupBy((stream: Stream) => stream.getId())
const guides = new Collection(data.guides).map(data => new Guide(data))
const guidesGroupedByStreamId = guides.groupBy((guide: Guide) => guide.getStreamId())
regions = regions.map((region: Region) =>
region
.withCountries(countriesKeyByCode)
.withRegions(regions)
.withSubdivisions(subdivisions)
.withCities(cities)
)
regionsKeyByCode = regions.keyBy((region: Region) => region.code)
countries = countries.map((country: Country) =>
country
.withCities(citiesGroupedByCountryCode)
.withSubdivisions(subdivisionsGroupedByCountryCode)
.withRegions(regions)
.withLanguage(languagesKeyByCode)
)
countriesKeyByCode = countries.keyBy((country: Country) => country.code)
subdivisions = subdivisions.map((subdivision: Subdivision) =>
subdivision
.withCities(citiesGroupedBySubdivisionCode)
.withCountry(countriesKeyByCode)
.withRegions(regions)
.withParent(subdivisionsKeyByCode)
)
subdivisionsKeyByCode = subdivisions.keyBy((subdivision: Subdivision) => subdivision.code)
subdivisionsGroupedByCountryCode = subdivisions.groupBy(
(subdivision: Subdivision) => subdivision.countryCode
)
channels = channels.map((channel: Channel) =>
channel
.withFeeds(feedsGroupedByChannelId)
.withLogos(logosGroupedByChannelId)
.withCategories(categoriesKeyById)
.withCountry(countriesKeyByCode)
.withSubdivision(subdivisionsKeyByCode)
.withCategories(categoriesKeyById)
)
channelsKeyById = channels.keyBy((channel: Channel) => channel.id)
feeds = feeds.map((feed: Feed) =>
feed
.withChannel(channelsKeyById)
.withLanguages(languagesKeyByCode)
.withTimezones(timezonesKeyById)
.withBroadcastArea(
citiesKeyByCode,
subdivisionsKeyByCode,
countriesKeyByCode,
regionsKeyByCode
)
)
feedsGroupedByChannelId = feeds.groupBy((feed: Feed) => feed.channelId)
feedsGroupedById = feeds.groupBy((feed: Feed) => feed.id)
return {
blocklistRecordsGroupedByChannelId,
subdivisionsGroupedByCountryCode,
feedsGroupedByChannelId,
guidesGroupedByStreamId,
logosGroupedByStreamId,
subdivisionsKeyByCode,
countriesKeyByCode,
languagesKeyByCode,
streamsGroupedById,
categoriesKeyById,
timezonesKeyById,
regionsKeyByCode,
blocklistRecords,
channelsKeyById,
citiesKeyByCode,
subdivisions,
categories,
countries,
languages,
timezones,
channels,
regions,
streams,
cities,
guides,
feeds,
logos
}
}
}

96
scripts/core/htmlTable.ts
View File

@@ -1,46 +1,50 @@
type Column = { import { Collection } from '@freearhey/core'
name: string
nowrap?: boolean export type HTMLTableColumn = {
align?: string name: string
} nowrap?: boolean
align?: string
type DataItem = string[] }
export class HTMLTable { export type HTMLTableItem = string[]
data: DataItem[]
columns: Column[] export class HTMLTable {
data: Collection<HTMLTableItem>
constructor(data: DataItem[], columns: Column[]) { columns: Collection<HTMLTableColumn>
this.data = data
this.columns = columns constructor(data: Collection<HTMLTableItem>, columns: Collection<HTMLTableColumn>) {
} this.data = data
this.columns = columns
toString() { }
let output = '<table>\r\n'
toString() {
output += ' <thead>\r\n <tr>' let output = '<table>\r\n'
for (const column of this.columns) {
output += `<th align="left">${column.name}</th>` output += ' <thead>\r\n <tr>'
} this.columns.forEach((column: HTMLTableColumn) => {
output += '</tr>\r\n </thead>\r\n' output += `<th align="left">${column.name}</th>`
})
output += ' <tbody>\r\n'
for (const item of this.data) { output += '</tr>\r\n </thead>\r\n'
output += ' <tr>'
let i = 0 output += ' <tbody>\r\n'
for (const prop in item) { this.data.forEach((item: HTMLTableItem) => {
const column = this.columns[i] output += ' <tr>'
const nowrap = column.nowrap ? ' nowrap' : '' let i = 0
const align = column.align ? ` align="${column.align}"` : '' for (const prop in item) {
output += `<td${align}${nowrap}>${item[prop]}</td>` const column = this.columns.all()[i]
i++ const nowrap = column.nowrap ? ' nowrap' : ''
} const align = column.align ? ` align="${column.align}"` : ''
output += '</tr>\r\n' output += `<td${align}${nowrap}>${item[prop]}</td>`
} i++
output += ' </tbody>\r\n' }
output += '</tr>\r\n'
output += '</table>' })
return output output += ' </tbody>\r\n'
}
} output += '</table>'
return output
}
}

25
scripts/core/index.ts
View File

@@ -1,14 +1,11 @@
export * from './apiClient' export * from './cliTable'
export * from './cliTable' export * from './htmlTable'
export * from './dataProcessor' export * from './issueData'
export * from './dataLoader' export * from './issueLoader'
export * from './htmlTable' export * from './issueParser'
export * from './issueData' export * from './logParser'
export * from './issueLoader' export * from './markdown'
export * from './issueParser' export * from './numberParser'
export * from './logParser' export * from './playlistParser'
export * from './markdown' export * from './proxyParser'
export * from './numberParser' export * from './streamTester'
export * from './playlistParser'
export * from './proxyParser'
export * from './streamTester'

70
scripts/core/issueData.ts
View File

@@ -1,34 +1,36 @@
import { Dictionary } from '@freearhey/core' import { Dictionary } from '@freearhey/core'
export class IssueData { export class IssueData {
_data: Dictionary _data: Dictionary<string>
constructor(data: Dictionary) { constructor(data: Dictionary<string>) {
this._data = data this._data = data
} }
has(key: string): boolean { has(key: string): boolean {
return this._data.has(key) return this._data.has(key)
} }
missing(key: string): boolean { missing(key: string): boolean {
return this._data.missing(key) || this._data.get(key) === undefined return this._data.missing(key) || this._data.get(key) === undefined
} }
getBoolean(key: string): boolean { getBoolean(key: string): boolean {
return Boolean(this._data.get(key)) return Boolean(this._data.get(key))
} }
getString(key: string): string | undefined { getString(key: string): string | undefined {
const deleteSymbol = '~' const deleteSymbol = '~'
return this._data.get(key) === deleteSymbol ? '' : this._data.get(key) return this._data.get(key) === deleteSymbol ? '' : this._data.get(key)
} }
getArray(key: string): string[] | undefined { getArray(key: string): string[] | undefined {
const deleteSymbol = '~' const deleteSymbol = '~'
if (this._data.missing(key)) return undefined if (this._data.missing(key)) return undefined
return this._data.get(key) === deleteSymbol ? [] : this._data.get(key).split('\r\n') const value = this._data.get(key)
}
} return !value || value === deleteSymbol ? [] : value.split('\r\n')
}
}

74
scripts/core/issueLoader.ts
View File

@@ -1,37 +1,37 @@
import { Collection } from '@freearhey/core' import { restEndpointMethods } from '@octokit/plugin-rest-endpoint-methods'
import { restEndpointMethods } from '@octokit/plugin-rest-endpoint-methods' import { paginateRest } from '@octokit/plugin-paginate-rest'
import { paginateRest } from '@octokit/plugin-paginate-rest' import { TESTING, OWNER, REPO } from '../constants'
import { Octokit } from '@octokit/core' import { Collection } from '@freearhey/core'
import { IssueParser } from './' import { Octokit } from '@octokit/core'
import { TESTING, OWNER, REPO } from '../constants' import { IssueParser } from './'
const CustomOctokit = Octokit.plugin(paginateRest, restEndpointMethods) const CustomOctokit = Octokit.plugin(paginateRest, restEndpointMethods)
const octokit = new CustomOctokit() const octokit = new CustomOctokit()
export class IssueLoader { export class IssueLoader {
async load(props?: { labels: string | string[] }) { async load(props?: { labels: string | string[] }) {
let labels = '' let labels = ''
if (props && props.labels) { if (props && props.labels) {
labels = Array.isArray(props.labels) ? props.labels.join(',') : props.labels labels = Array.isArray(props.labels) ? props.labels.join(',') : props.labels
} }
let issues: object[] = [] let issues: object[] = []
if (TESTING) { if (TESTING) {
issues = (await import('../../tests/__data__/input/issues.js')).default issues = (await import('../../tests/__data__/input/issues.js')).default
} else { } else {
issues = await octokit.paginate(octokit.rest.issues.listForRepo, { issues = await octokit.paginate(octokit.rest.issues.listForRepo, {
owner: OWNER, owner: OWNER,
repo: REPO, repo: REPO,
per_page: 100, per_page: 100,
labels, labels,
status: 'open', status: 'open',
headers: { headers: {
'X-GitHub-Api-Version': '2022-11-28' 'X-GitHub-Api-Version': '2022-11-28'
} }
}) })
} }
const parser = new IssueParser() const parser = new IssueParser()
return new Collection(issues).map(parser.parse) return new Collection(issues).map(parser.parse)
} }
} }

96
scripts/core/issueParser.ts
View File

@@ -1,48 +1,48 @@
import { Dictionary } from '@freearhey/core' import { Dictionary } from '@freearhey/core'
import { Issue } from '../models' import { IssueData } from './issueData'
import { IssueData } from './issueData' import { Issue } from '../models'
const FIELDS = new Dictionary({ const FIELDS = new Dictionary({
'Stream ID': 'streamId', 'Stream ID': 'streamId',
'Channel ID': 'channelId', 'Channel ID': 'channelId',
'Feed ID': 'feedId', 'Feed ID': 'feedId',
'Stream URL': 'streamUrl', 'Stream URL': 'streamUrl',
'New Stream URL': 'newStreamUrl', 'New Stream URL': 'newStreamUrl',
Label: 'label', Label: 'label',
Quality: 'quality', Quality: 'quality',
'HTTP User-Agent': 'httpUserAgent', 'HTTP User-Agent': 'httpUserAgent',
'HTTP User Agent': 'httpUserAgent', 'HTTP User Agent': 'httpUserAgent',
'HTTP Referrer': 'httpReferrer', 'HTTP Referrer': 'httpReferrer',
'What happened to the stream?': 'reason', 'What happened to the stream?': 'reason',
Reason: 'reason', Reason: 'reason',
Notes: 'notes', Notes: 'notes',
Directives: 'directives' Directives: 'directives'
}) })
export class IssueParser { export class IssueParser {
parse(issue: { number: number; body: string; labels: { name: string }[] }): Issue { parse(issue: { number: number; body: string; labels: { name: string }[] }): Issue {
const fields = typeof issue.body === 'string' ? issue.body.split('###') : [] const fields = typeof issue.body === 'string' ? issue.body.split('###') : []
const data = new Dictionary() const data = new Dictionary<string>()
fields.forEach((field: string) => { fields.forEach((field: string) => {
const parsed = typeof field === 'string' ? field.split(/\r?\n/).filter(Boolean) : [] const parsed = typeof field === 'string' ? field.split(/\r?\n/).filter(Boolean) : []
let _label = parsed.shift() let _label = parsed.shift()
_label = _label ? _label.replace(/ \(optional\)| \(required\)/, '').trim() : '' _label = _label ? _label.replace(/ \(optional\)| \(required\)/, '').trim() : ''
let _value = parsed.join('\r\n') let _value = parsed.join('\r\n')
_value = _value ? _value.trim() : '' _value = _value ? _value.trim() : ''
if (!_label || !_value) return data if (!_label || !_value) return data
const id: string = FIELDS.get(_label) const id = FIELDS.get(_label)
const value: string = _value === '_No response_' || _value === 'None' ? '' : _value const value: string = _value === '_No response_' || _value === 'None' ? '' : _value
if (!id) return if (!id) return
data.set(id, value) data.set(id, value)
}) })
const labels = issue.labels.map(label => label.name) const labels = issue.labels.map(label => label.name)
return new Issue({ number: issue.number, labels, data: new IssueData(data) }) return new Issue({ number: issue.number, labels, data: new IssueData(data) })
} }
} }

28
scripts/core/logParser.ts
View File

@@ -1,14 +1,14 @@
export type LogItem = { export type LogItem = {
type: string type: string
filepath: string filepath: string
count: number count: number
} }
export class LogParser { export class LogParser {
parse(content: string): LogItem[] { parse(content: string): LogItem[] {
if (!content) return [] if (!content) return []
const lines = content.split('\n') const lines = content.split('\n')
return lines.map(line => (line ? JSON.parse(line) : null)).filter(l => l) return lines.map(line => (line ? JSON.parse(line) : null)).filter(l => l)
} }
} }

90
scripts/core/markdown.ts
View File

@@ -1,45 +1,45 @@
import fs from 'fs' import path from 'path'
import path from 'path' import fs from 'fs'
type MarkdownConfig = { type MarkdownConfig = {
build: string build: string
template: string template: string
} }
export class Markdown { export class Markdown {
build: string build: string
template: string template: string
constructor(config: MarkdownConfig) { constructor(config: MarkdownConfig) {
this.build = config.build this.build = config.build
this.template = config.template this.template = config.template
} }
compile() { compile() {
const workingDir = process.cwd() const workingDir = process.cwd()
const templatePath = path.resolve(workingDir, this.template) const templatePath = path.resolve(workingDir, this.template)
const template = fs.readFileSync(templatePath, 'utf8') const template = fs.readFileSync(templatePath, 'utf8')
const processedContent = this.processIncludes(template, workingDir) const processedContent = this.processIncludes(template, workingDir)
if (this.build) { if (this.build) {
const outputPath = path.resolve(workingDir, this.build) const outputPath = path.resolve(workingDir, this.build)
fs.writeFileSync(outputPath, processedContent, 'utf8') fs.writeFileSync(outputPath, processedContent, 'utf8')
} }
} }
private processIncludes(template: string, baseDir: string): string { private processIncludes(template: string, baseDir: string): string {
const includeRegex = /#include\s+"([^"]+)"/g const includeRegex = /#include\s+"([^"]+)"/g
return template.replace(includeRegex, (match, includePath) => { return template.replace(includeRegex, (match, includePath) => {
try { try {
const fullPath = path.resolve(baseDir, includePath) const fullPath = path.resolve(baseDir, includePath)
const includeContent = fs.readFileSync(fullPath, 'utf8') const includeContent = fs.readFileSync(fullPath, 'utf8')
return this.processIncludes(includeContent, baseDir) return this.processIncludes(includeContent, baseDir)
} catch (error) { } catch (error) {
console.warn(`Warning: Could not include file ${includePath}: ${error}`) console.warn(`Warning: Could not include file ${includePath}: ${error}`)
return match return match
} }
}) })
} }
} }

20
scripts/core/numberParser.ts
View File

@@ -1,10 +1,10 @@
export default class NumberParser { export default class NumberParser {
async parse(number: string) { async parse(number: string) {
const parsed = parseInt(number) const parsed = parseInt(number)
if (isNaN(parsed)) { if (isNaN(parsed)) {
throw new Error('numberParser:parse() Input value is not a number') throw new Error('numberParser:parse() Input value is not a number')
} }
return parsed return parsed
} }
} }

103
scripts/core/playlistParser.ts
View File

@@ -1,60 +1,43 @@
import { Collection, Storage, Dictionary } from '@freearhey/core' import { Storage } from '@freearhey/storage-js'
import parser from 'iptv-playlist-parser' import { Collection } from '@freearhey/core'
import { Stream } from '../models' import parser from 'iptv-playlist-parser'
import { Stream } from '../models'
type PlaylistPareserProps = {
storage: Storage type PlaylistPareserProps = {
feedsGroupedByChannelId: Dictionary storage: Storage
logosGroupedByStreamId: Dictionary }
channelsKeyById: Dictionary
} export class PlaylistParser {
storage: Storage
export class PlaylistParser {
storage: Storage constructor({ storage }: PlaylistPareserProps) {
feedsGroupedByChannelId: Dictionary this.storage = storage
logosGroupedByStreamId: Dictionary }
channelsKeyById: Dictionary
async parse(files: string[]): Promise<Collection<Stream>> {
constructor({ const parsed = new Collection<Stream>()
storage,
feedsGroupedByChannelId, for (const filepath of files) {
logosGroupedByStreamId, if (!this.storage.existsSync(filepath)) continue
channelsKeyById const _parsed: Collection<Stream> = await this.parseFile(filepath)
}: PlaylistPareserProps) { parsed.concat(_parsed)
this.storage = storage }
this.feedsGroupedByChannelId = feedsGroupedByChannelId
this.logosGroupedByStreamId = logosGroupedByStreamId return parsed
this.channelsKeyById = channelsKeyById }
}
async parseFile(filepath: string): Promise<Collection<Stream>> {
async parse(files: string[]): Promise<Collection> { const content = await this.storage.load(filepath)
let streams = new Collection() const parsed: parser.Playlist = parser.parse(content)
for (const filepath of files) { const streams = new Collection<Stream>()
if (!this.storage.existsSync(filepath)) continue parsed.items.forEach((data: parser.PlaylistItem) => {
const stream = Stream.fromPlaylistItem(data)
const _streams: Collection = await this.parseFile(filepath) stream.filepath = filepath
streams = streams.concat(_streams)
} streams.add(stream)
})
return streams
} return streams
}
async parseFile(filepath: string): Promise<Collection> { }
const content = await this.storage.load(filepath)
const parsed: parser.Playlist = parser.parse(content)
const streams = new Collection(parsed.items).map((data: parser.PlaylistItem) => {
const stream = new Stream()
.fromPlaylistItem(data)
.withFeed(this.feedsGroupedByChannelId)
.withChannel(this.channelsKeyById)
.withLogos(this.logosGroupedByStreamId)
.setFilepath(filepath)
return stream
})
return streams
}
}

62
scripts/core/proxyParser.ts
View File

@@ -1,31 +1,31 @@
import { URL } from 'node:url' import { URL } from 'node:url'
interface ProxyParserResult { interface ProxyParserResult {
protocol: string | null protocol: string | null
auth?: { auth?: {
username?: string username?: string
password?: string password?: string
} }
host: string host: string
port: number | null port: number | null
} }
export class ProxyParser { export class ProxyParser {
parse(_url: string): ProxyParserResult { parse(_url: string): ProxyParserResult {
const parsed = new URL(_url) const parsed = new URL(_url)
const result: ProxyParserResult = { const result: ProxyParserResult = {
protocol: parsed.protocol.replace(':', '') || null, protocol: parsed.protocol.replace(':', '') || null,
host: parsed.hostname, host: parsed.hostname,
port: parsed.port ? parseInt(parsed.port) : null port: parsed.port ? parseInt(parsed.port) : null
} }
if (parsed.username || parsed.password) { if (parsed.username || parsed.password) {
result.auth = {} result.auth = {}
if (parsed.username) result.auth.username = parsed.username if (parsed.username) result.auth.username = parsed.username
if (parsed.password) result.auth.password = parsed.password if (parsed.password) result.auth.password = parsed.password
} }
return result return result
} }
} }

242
scripts/core/streamTester.ts
View File

@@ -1,117 +1,125 @@
import { Stream } from '../models' import axios, { AxiosInstance, AxiosProxyConfig, AxiosRequestConfig, AxiosResponse } from 'axios'
import { TESTING } from '../constants' import { SocksProxyAgent } from 'socks-proxy-agent'
import mediaInfoFactory from 'mediainfo.js' import { ProxyParser } from './proxyParser.js'
import axios, { AxiosInstance, AxiosProxyConfig, AxiosRequestConfig } from 'axios' import mediaInfoFactory from 'mediainfo.js'
import { ProxyParser } from './proxyParser.js' import { OptionValues } from 'commander'
import { OptionValues } from 'commander' import { TESTING } from '../constants'
import { SocksProxyAgent } from 'socks-proxy-agent' import { Stream } from '../models'
export type TestResult = { export type StreamTesterResult = {
status: { status: {
ok: boolean ok: boolean
code: string code: string
} }
} }
export type StreamTesterProps = { export type StreamTesterError = {
options: OptionValues name: string
} code?: string
cause?: Error & { code?: string }
export class StreamTester { response?: AxiosResponse
client: AxiosInstance }
options: OptionValues
export type StreamTesterProps = {
constructor({ options }: StreamTesterProps) { options: OptionValues
const proxyParser = new ProxyParser() }
let request: AxiosRequestConfig = {
responseType: 'arraybuffer' export class StreamTester {
} client: AxiosInstance
options: OptionValues
if (options.proxy !== undefined) {
const proxy = proxyParser.parse(options.proxy) as AxiosProxyConfig constructor({ options }: StreamTesterProps) {
const proxyParser = new ProxyParser()
if ( let request: AxiosRequestConfig = {
proxy.protocol && responseType: 'arraybuffer'
['socks', 'socks5', 'socks5h', 'socks4', 'socks4a'].includes(String(proxy.protocol)) }
) {
const socksProxyAgent = new SocksProxyAgent(options.proxy) if (options.proxy !== undefined) {
const proxy = proxyParser.parse(options.proxy) as AxiosProxyConfig
request = { ...request, ...{ httpAgent: socksProxyAgent, httpsAgent: socksProxyAgent } }
} else { if (
request = { ...request, ...{ proxy } } proxy.protocol &&
} ['socks', 'socks5', 'socks5h', 'socks4', 'socks4a'].includes(String(proxy.protocol))
} ) {
const socksProxyAgent = new SocksProxyAgent(options.proxy)
this.client = axios.create(request)
this.options = options request = { ...request, ...{ httpAgent: socksProxyAgent, httpsAgent: socksProxyAgent } }
} } else {
request = { ...request, ...{ proxy } }
async test(stream: Stream): Promise<TestResult> { }
if (TESTING) { }
const results = (await import('../../tests/__data__/input/playlist_test/results.js')).default
this.client = axios.create(request)
return results[stream.url as keyof typeof results] this.options = options
} else { }
try {
const res = await this.client(stream.url, { async test(stream: Stream): Promise<StreamTesterResult> {
signal: AbortSignal.timeout(this.options.timeout), if (TESTING) {
headers: { const results = (await import('../../tests/__data__/input/playlist_test/results.js')).default
'User-Agent': stream.getUserAgent() || 'Mozilla/5.0',
Referer: stream.getReferrer() return results[stream.url as keyof typeof results]
} } else {
}) try {
const res = await this.client(stream.url, {
const mediainfo = await mediaInfoFactory({ format: 'object' }) signal: AbortSignal.timeout(this.options.timeout),
const buffer = await res.data headers: {
const result = await mediainfo.analyzeData( 'User-Agent': stream.user_agent || 'Mozilla/5.0',
() => buffer.byteLength, Referer: stream.referrer
(size: any, offset: number | undefined) => }
Buffer.from(buffer).subarray(offset, offset + size) })
)
const mediainfo = await mediaInfoFactory({ format: 'object' })
if (result && result.media && result.media.track.length > 0) { const buffer = await res.data
return { const result = await mediainfo.analyzeData(
status: { () => buffer.byteLength,
ok: true, (size: number, offset: number) => Buffer.from(buffer).subarray(offset, offset + size)
code: 'OK' )
}
} if (result && result.media && result.media.track.length > 0) {
} else { return {
return { status: {
status: { ok: true,
ok: false, code: 'OK'
code: 'NO_VIDEO' }
} }
} } else {
} return {
} catch (error: any) { status: {
let code = 'UNKNOWN_ERROR' ok: false,
if (error.name === 'CanceledError') { code: 'NO_VIDEO'
code = 'TIMEOUT' }
} else if (error.name === 'AxiosError') { }
if (error.response) { }
const status = error.response?.status } catch (err: unknown) {
const statusText = error.response?.statusText.toUpperCase().replace(/\s+/, '_') const error = err as StreamTesterError
code = `HTTP_${status}_${statusText}`
} else { let code = 'UNKNOWN_ERROR'
code = `AXIOS_${error.code}` if (error.name === 'CanceledError') {
} code = 'TIMEOUT'
} else if (error.cause) { } else if (error.name === 'AxiosError') {
const cause = error.cause as Error & { code?: string } if (error.response) {
if (cause.code) { const status = error.response?.status
code = cause.code const statusText = error.response?.statusText.toUpperCase().replace(/\s+/, '_')
} else { code = `HTTP_${status}_${statusText}`
code = cause.name } else {
} code = `AXIOS_${error.code}`
} }
} else if (error.cause) {
return { const cause = error.cause
status: { if (cause.code) {
ok: false, code = cause.code
code } else {
} code = cause.name
} }
} }
}
} return {
} status: {
ok: false,
code
}
}
}
}
}
}

114
scripts/generators/categoriesGenerator.ts
View File

@@ -1,54 +1,60 @@
import { Collection, Storage, File } from '@freearhey/core' import { Storage, File } from '@freearhey/storage-js'
import { Stream, Category, Playlist } from '../models' import { PUBLIC_DIR, EOL } from '../constants'
import { PUBLIC_DIR, EOL } from '../constants' import { Collection } from '@freearhey/core'
import { Generator } from './generator' import { Stream, Playlist } from '../models'
import { Generator } from './generator'
type CategoriesGeneratorProps = { import * as sdk from '@iptv-org/sdk'
streams: Collection
categories: Collection type CategoriesGeneratorProps = {
logFile: File streams: Collection<Stream>
} categories: Collection<sdk.Models.Category>
logFile: File
export class CategoriesGenerator implements Generator { }
streams: Collection
categories: Collection export class CategoriesGenerator implements Generator {
storage: Storage streams: Collection<Stream>
logFile: File categories: Collection<sdk.Models.Category>
storage: Storage
constructor({ streams, categories, logFile }: CategoriesGeneratorProps) { logFile: File
this.streams = streams.clone()
this.categories = categories constructor({ streams, categories, logFile }: CategoriesGeneratorProps) {
this.storage = new Storage(PUBLIC_DIR) this.streams = streams.clone()
this.logFile = logFile this.categories = categories
} this.storage = new Storage(PUBLIC_DIR)
this.logFile = logFile
async generate() { }
const streams = this.streams.orderBy([(stream: Stream) => stream.getTitle()])
async generate() {
this.categories.forEach(async (category: Category) => { const streams = this.streams.sortBy([(stream: Stream) => stream.title])
const categoryStreams = streams
.filter((stream: Stream) => stream.hasCategory(category)) this.categories.forEach(async (category: sdk.Models.Category) => {
.map((stream: Stream) => { const categoryStreams = streams
const groupTitle = stream.getCategoryNames().join(';') .filter((stream: Stream) => stream.hasCategory(category))
if (groupTitle) stream.groupTitle = groupTitle .map((stream: Stream) => {
const groupTitle = stream
return stream .getCategories()
}) .map(category => category.name)
.sort()
const playlist = new Playlist(categoryStreams, { public: true }) .join(';')
const filepath = `categories/${category.id}.m3u` if (groupTitle) stream.groupTitle = groupTitle
await this.storage.save(filepath, playlist.toString())
this.logFile.append( return stream
JSON.stringify({ type: 'category', filepath, count: playlist.streams.count() }) + EOL })
)
}) const playlist = new Playlist(categoryStreams, { public: true })
const filepath = `categories/${category.id}.m3u`
const undefinedStreams = streams.filter((stream: Stream) => !stream.hasCategories()) await this.storage.save(filepath, playlist.toString())
const playlist = new Playlist(undefinedStreams, { public: true }) this.logFile.append(
const filepath = 'categories/undefined.m3u' JSON.stringify({ type: 'category', filepath, count: playlist.streams.count() }) + EOL
await this.storage.save(filepath, playlist.toString()) )
this.logFile.append( })
JSON.stringify({ type: 'category', filepath, count: playlist.streams.count() }) + EOL
) const undefinedStreams = streams.filter((stream: Stream) => stream.getCategories().isEmpty())
} const playlist = new Playlist(undefinedStreams, { public: true })
} const filepath = 'categories/undefined.m3u'
await this.storage.save(filepath, playlist.toString())
this.logFile.append(
JSON.stringify({ type: 'category', filepath, count: playlist.streams.count() }) + EOL
)
}
}

97
scripts/generators/citiesGenerator.ts
View File

@@ -1,43 +1,54 @@
import { City, Stream, Playlist } from '../models' import { Storage, File } from '@freearhey/storage-js'
import { Collection, Storage, File } from '@freearhey/core' import { PUBLIC_DIR, EOL } from '../constants'
import { PUBLIC_DIR, EOL } from '../constants' import { Stream, Playlist } from '../models'
import { Generator } from './generator' import { Collection } from '@freearhey/core'
import { Generator } from './generator'
type CitiesGeneratorProps = { import * as sdk from '@iptv-org/sdk'
streams: Collection
cities: Collection type CitiesGeneratorProps = {
logFile: File streams: Collection<Stream>
} cities: Collection<sdk.Models.City>
logFile: File
export class CitiesGenerator implements Generator { }
streams: Collection
cities: Collection export class CitiesGenerator implements Generator {
storage: Storage streams: Collection<Stream>
logFile: File cities: Collection<sdk.Models.City>
storage: Storage
constructor({ streams, cities, logFile }: CitiesGeneratorProps) { logFile: File
this.streams = streams.clone()
this.cities = cities constructor({ streams, cities, logFile }: CitiesGeneratorProps) {
this.storage = new Storage(PUBLIC_DIR) this.streams = streams.clone()
this.logFile = logFile this.cities = cities
} this.storage = new Storage(PUBLIC_DIR)
this.logFile = logFile
async generate(): Promise<void> { }
const streams = this.streams
.orderBy((stream: Stream) => stream.getTitle()) async generate(): Promise<void> {
.filter((stream: Stream) => stream.isSFW()) const streams = this.streams
.sortBy((stream: Stream) => stream.title)
this.cities.forEach(async (city: City) => { .filter((stream: Stream) => stream.isSFW())
const cityStreams = streams.filter((stream: Stream) => stream.isBroadcastInCity(city))
const streamsGroupedByCityCode = {}
if (cityStreams.isEmpty()) return streams.forEach((stream: Stream) => {
stream.getBroadcastCities().forEach((city: sdk.Models.City) => {
const playlist = new Playlist(cityStreams, { public: true }) if (streamsGroupedByCityCode[city.code]) {
const filepath = `cities/${city.code.toLowerCase()}.m3u` streamsGroupedByCityCode[city.code].add(stream)
await this.storage.save(filepath, playlist.toString()) } else {
this.logFile.append( streamsGroupedByCityCode[city.code] = new Collection<Stream>([stream])
JSON.stringify({ type: 'city', filepath, count: playlist.streams.count() }) + EOL }
) })
}) })
}
} for (const cityCode in streamsGroupedByCityCode) {
const cityStreams = streamsGroupedByCityCode[cityCode]
const playlist = new Playlist(cityStreams, { public: true })
const filepath = `cities/${cityCode.toLowerCase()}.m3u`
await this.storage.save(filepath, playlist.toString())
this.logFile.append(
JSON.stringify({ type: 'city', filepath, count: playlist.streams.count() }) + EOL
)
}
}
}

148
scripts/generators/countriesGenerator.ts
View File

@@ -1,68 +1,80 @@
import { Country, Stream, Playlist } from '../models' import { Storage, File } from '@freearhey/storage-js'
import { Collection, Storage, File } from '@freearhey/core' import { PUBLIC_DIR, EOL } from '../constants'
import { PUBLIC_DIR, EOL } from '../constants' import { Stream, Playlist } from '../models'
import { Generator } from './generator' import { Collection } from '@freearhey/core'
import { Generator } from './generator'
type CountriesGeneratorProps = { import * as sdk from '@iptv-org/sdk'
streams: Collection
countries: Collection type CountriesGeneratorProps = {
logFile: File streams: Collection<Stream>
} countries: Collection<sdk.Models.Country>
logFile: File
export class CountriesGenerator implements Generator { }
streams: Collection
countries: Collection export class CountriesGenerator implements Generator {
storage: Storage streams: Collection<Stream>
logFile: File countries: Collection<sdk.Models.Country>
storage: Storage
constructor({ streams, countries, logFile }: CountriesGeneratorProps) { logFile: File
this.streams = streams.clone()
this.countries = countries constructor({ streams, countries, logFile }: CountriesGeneratorProps) {
this.storage = new Storage(PUBLIC_DIR) this.streams = streams.clone()
this.logFile = logFile this.countries = countries
} this.storage = new Storage(PUBLIC_DIR)
this.logFile = logFile
async generate(): Promise<void> { }
const streams = this.streams
.orderBy((stream: Stream) => stream.getTitle()) async generate(): Promise<void> {
.filter((stream: Stream) => stream.isSFW()) const streams = this.streams
.sortBy((stream: Stream) => stream.title)
this.countries.forEach(async (country: Country) => { .filter((stream: Stream) => stream.isSFW())
const countryStreams = streams.filter((stream: Stream) =>
stream.isBroadcastInCountry(country) const streamsGroupedByCountryCode = {}
) streams.forEach((stream: Stream) => {
if (countryStreams.isEmpty()) return stream.getBroadcastCountries().forEach((country: sdk.Models.Country) => {
if (streamsGroupedByCountryCode[country.code]) {
const playlist = new Playlist(countryStreams, { public: true }) streamsGroupedByCountryCode[country.code].add(stream)
const filepath = `countries/${country.code.toLowerCase()}.m3u` } else {
await this.storage.save(filepath, playlist.toString()) streamsGroupedByCountryCode[country.code] = new Collection<Stream>([stream])
this.logFile.append( }
JSON.stringify({ type: 'country', filepath, count: playlist.streams.count() }) + EOL })
) })
})
for (const countryCode in streamsGroupedByCountryCode) {
const internationalStreams = streams.filter((stream: Stream) => stream.isInternational()) const countryStreams = streamsGroupedByCountryCode[countryCode]
const internationalPlaylist = new Playlist(internationalStreams, { public: true })
const internationalFilepath = 'countries/int.m3u' const playlist = new Playlist(countryStreams, { public: true })
await this.storage.save(internationalFilepath, internationalPlaylist.toString()) const filepath = `countries/${countryCode.toLowerCase()}.m3u`
this.logFile.append( await this.storage.save(filepath, playlist.toString())
JSON.stringify({ this.logFile.append(
type: 'country', JSON.stringify({ type: 'country', filepath, count: playlist.streams.count() }) + EOL
filepath: internationalFilepath, )
count: internationalPlaylist.streams.count() }
}) + EOL
) const internationalStreams = streams.filter((stream: Stream) => stream.isInternational())
const internationalPlaylist = new Playlist(internationalStreams, { public: true })
const undefinedStreams = streams.filter((stream: Stream) => !stream.hasBroadcastArea()) const internationalFilepath = 'countries/int.m3u'
const undefinedPlaylist = new Playlist(undefinedStreams, { public: true }) await this.storage.save(internationalFilepath, internationalPlaylist.toString())
const undefinedFilepath = 'countries/undefined.m3u' this.logFile.append(
await this.storage.save(undefinedFilepath, undefinedPlaylist.toString()) JSON.stringify({
this.logFile.append( type: 'country',
JSON.stringify({ filepath: internationalFilepath,
type: 'country', count: internationalPlaylist.streams.count()
filepath: undefinedFilepath, }) + EOL
count: undefinedPlaylist.streams.count() )
}) + EOL
) const undefinedStreams = streams.filter((stream: Stream) =>
} stream.getBroadcastAreaCodes().isEmpty()
} )
const undefinedPlaylist = new Playlist(undefinedStreams, { public: true })
const undefinedFilepath = 'countries/undefined.m3u'
await this.storage.save(undefinedFilepath, undefinedPlaylist.toString())
this.logFile.append(
JSON.stringify({
type: 'country',
filepath: undefinedFilepath,
count: undefinedPlaylist.streams.count()
}) + EOL
)
}
}

6
scripts/generators/generator.ts
View File

@@ -1,3 +1,3 @@
export interface Generator { export interface Generator {
generate(): Promise<void> generate(): Promise<void>
} }

25
scripts/generators/index.ts
View File

@@ -1,13 +1,12 @@
export * from './categoriesGenerator' export * from './categoriesGenerator'
export * from './citiesGenerator' export * from './citiesGenerator'
export * from './countriesGenerator' export * from './countriesGenerator'
export * from './indexCategoryGenerator' export * from './indexCategoryGenerator'
export * from './indexCountryGenerator' export * from './indexCountryGenerator'
export * from './indexGenerator' export * from './indexGenerator'
export * from './indexLanguageGenerator' export * from './indexLanguageGenerator'
export * from './indexNsfwGenerator' export * from './languagesGenerator'
export * from './languagesGenerator' export * from './rawGenerator'
export * from './rawGenerator' export * from './regionsGenerator'
export * from './regionsGenerator' export * from './sourcesGenerator'
export * from './sourcesGenerator' export * from './subdivisionsGenerator'
export * from './subdivisionsGenerator'

111
scripts/generators/indexCategoryGenerator.ts
View File

@@ -1,55 +1,56 @@
import { Collection, Storage, File } from '@freearhey/core' import { Storage, File } from '@freearhey/storage-js'
import { Stream, Playlist, Category } from '../models' import { PUBLIC_DIR, EOL } from '../constants'
import { PUBLIC_DIR, EOL } from '../constants' import { Stream, Playlist } from '../models'
import { Generator } from './generator' import { Collection } from '@freearhey/core'
import { Generator } from './generator'
type IndexCategoryGeneratorProps = { import * as sdk from '@iptv-org/sdk'
streams: Collection
logFile: File type IndexCategoryGeneratorProps = {
} streams: Collection<Stream>
logFile: File
export class IndexCategoryGenerator implements Generator { }
streams: Collection
storage: Storage export class IndexCategoryGenerator implements Generator {
logFile: File streams: Collection<Stream>
storage: Storage
constructor({ streams, logFile }: IndexCategoryGeneratorProps) { logFile: File
this.streams = streams.clone()
this.storage = new Storage(PUBLIC_DIR) constructor({ streams, logFile }: IndexCategoryGeneratorProps) {
this.logFile = logFile this.streams = streams.clone()
} this.storage = new Storage(PUBLIC_DIR)
this.logFile = logFile
async generate(): Promise<void> { }
const streams = this.streams
.orderBy(stream => stream.getTitle()) async generate(): Promise<void> {
.filter(stream => stream.isSFW()) const streams = this.streams.sortBy(stream => stream.title).filter(stream => stream.isSFW())
let groupedStreams = new Collection() let groupedStreams = new Collection<Stream>()
streams.forEach((stream: Stream) => { streams.forEach((stream: Stream) => {
if (!stream.hasCategories()) { const streamCategories = stream.getCategories()
const streamClone = stream.clone() if (streamCategories.isEmpty()) {
streamClone.groupTitle = 'Undefined' const streamClone = stream.clone()
groupedStreams.add(streamClone) streamClone.groupTitle = 'Undefined'
return groupedStreams.add(streamClone)
} return
}
stream.getCategories().forEach((category: Category) => {
const streamClone = stream.clone() streamCategories.forEach((category: sdk.Models.Category) => {
streamClone.groupTitle = category.name const streamClone = stream.clone()
groupedStreams.push(streamClone) streamClone.groupTitle = category.name
}) groupedStreams.add(streamClone)
}) })
})
groupedStreams = groupedStreams.orderBy(stream => {
if (stream.groupTitle === 'Undefined') return 'ZZ' groupedStreams = groupedStreams.sortBy(stream => {
return stream.groupTitle if (stream.groupTitle === 'Undefined') return 'ZZ'
}) return stream.groupTitle
})
const playlist = new Playlist(groupedStreams, { public: true })
const filepath = 'index.category.m3u' const playlist = new Playlist(groupedStreams, { public: true })
await this.storage.save(filepath, playlist.toString()) const filepath = 'index.category.m3u'
this.logFile.append( await this.storage.save(filepath, playlist.toString())
JSON.stringify({ type: 'index', filepath, count: playlist.streams.count() }) + EOL this.logFile.append(
) JSON.stringify({ type: 'index', filepath, count: playlist.streams.count() }) + EOL
} )
} }
}

130
scripts/generators/indexCountryGenerator.ts
View File

@@ -1,63 +1,67 @@
import { Collection, Storage, File } from '@freearhey/core' import { Storage, File } from '@freearhey/storage-js'
import { Stream, Playlist, Country } from '../models' import { PUBLIC_DIR, EOL } from '../constants'
import { PUBLIC_DIR, EOL } from '../constants' import { Stream, Playlist } from '../models'
import { Generator } from './generator' import { Collection } from '@freearhey/core'
import { Generator } from './generator'
type IndexCountryGeneratorProps = { import * as sdk from '@iptv-org/sdk'
streams: Collection
logFile: File type IndexCountryGeneratorProps = {
} streams: Collection<Stream>
logFile: File
export class IndexCountryGenerator implements Generator { }
streams: Collection
storage: Storage export class IndexCountryGenerator implements Generator {
logFile: File streams: Collection<Stream>
storage: Storage
constructor({ streams, logFile }: IndexCountryGeneratorProps) { logFile: File
this.streams = streams.clone()
this.storage = new Storage(PUBLIC_DIR) constructor({ streams, logFile }: IndexCountryGeneratorProps) {
this.logFile = logFile this.streams = streams.clone()
} this.storage = new Storage(PUBLIC_DIR)
this.logFile = logFile
async generate(): Promise<void> { }
let groupedStreams = new Collection()
async generate(): Promise<void> {
this.streams let groupedStreams = new Collection<Stream>()
.orderBy((stream: Stream) => stream.getTitle())
.filter((stream: Stream) => stream.isSFW()) this.streams
.forEach((stream: Stream) => { .sortBy((stream: Stream) => stream.title)
if (!stream.hasBroadcastArea()) { .filter((stream: Stream) => stream.isSFW())
const streamClone = stream.clone() .forEach((stream: Stream) => {
streamClone.groupTitle = 'Undefined' const broadcastAreaCountries = stream.getBroadcastCountries()
groupedStreams.add(streamClone)
return if (stream.getBroadcastAreaCodes().isEmpty()) {
} const streamClone = stream.clone()
streamClone.groupTitle = 'Undefined'
stream.getBroadcastCountries().forEach((country: Country) => { groupedStreams.add(streamClone)
const streamClone = stream.clone() return
streamClone.groupTitle = country.name }
groupedStreams.add(streamClone)
}) broadcastAreaCountries.forEach((country: sdk.Models.Country) => {
const streamClone = stream.clone()
if (stream.isInternational()) { streamClone.groupTitle = country.name
const streamClone = stream.clone() groupedStreams.add(streamClone)
streamClone.groupTitle = 'International' })
groupedStreams.add(streamClone)
} if (stream.isInternational()) {
}) const streamClone = stream.clone()
streamClone.groupTitle = 'International'
groupedStreams = groupedStreams.orderBy((stream: Stream) => { groupedStreams.add(streamClone)
if (stream.groupTitle === 'International') return 'ZZ' }
if (stream.groupTitle === 'Undefined') return 'ZZZ' })
return stream.groupTitle groupedStreams = groupedStreams.sortBy((stream: Stream) => {
}) if (stream.groupTitle === 'International') return 'ZZ'
if (stream.groupTitle === 'Undefined') return 'ZZZ'
const playlist = new Playlist(groupedStreams, { public: true })
const filepath = 'index.country.m3u' return stream.groupTitle
await this.storage.save(filepath, playlist.toString()) })
this.logFile.append(
JSON.stringify({ type: 'index', filepath, count: playlist.streams.count() }) + EOL const playlist = new Playlist(groupedStreams, { public: true })
) const filepath = 'index.country.m3u'
} await this.storage.save(filepath, playlist.toString())
} this.logFile.append(
JSON.stringify({ type: 'index', filepath, count: playlist.streams.count() }) + EOL
)
}
}

85
scripts/generators/indexGenerator.ts
View File

@@ -1,40 +1,45 @@
import { Collection, File, Storage } from '@freearhey/core' import { Storage, File } from '@freearhey/storage-js'
import { Stream, Playlist } from '../models' import { PUBLIC_DIR, EOL } from '../constants'
import { PUBLIC_DIR, EOL } from '../constants' import { Stream, Playlist } from '../models'
import { Generator } from './generator' import { Collection } from '@freearhey/core'
import { Generator } from './generator'
type IndexGeneratorProps = {
streams: Collection type IndexGeneratorProps = {
logFile: File streams: Collection<Stream>
} logFile: File
}
export class IndexGenerator implements Generator {
streams: Collection export class IndexGenerator implements Generator {
storage: Storage streams: Collection<Stream>
logFile: File storage: Storage
logFile: File
constructor({ streams, logFile }: IndexGeneratorProps) {
this.streams = streams.clone() constructor({ streams, logFile }: IndexGeneratorProps) {
this.storage = new Storage(PUBLIC_DIR) this.streams = streams.clone()
this.logFile = logFile this.storage = new Storage(PUBLIC_DIR)
} this.logFile = logFile
}
async generate(): Promise<void> {
const sfwStreams = this.streams async generate(): Promise<void> {
.orderBy(stream => stream.getTitle()) const sfwStreams = this.streams
.filter((stream: Stream) => stream.isSFW()) .sortBy(stream => stream.title)
.map((stream: Stream) => { .filter((stream: Stream) => stream.isSFW())
const groupTitle = stream.getCategoryNames().join(';') .map((stream: Stream) => {
if (groupTitle) stream.groupTitle = groupTitle const groupTitle = stream
.getCategories()
return stream .map(category => category.name)
}) .sort()
.join(';')
const playlist = new Playlist(sfwStreams, { public: true }) if (groupTitle) stream.groupTitle = groupTitle
const filepath = 'index.m3u'
await this.storage.save(filepath, playlist.toString()) return stream
this.logFile.append( })
JSON.stringify({ type: 'index', filepath, count: playlist.streams.count() }) + EOL
) const playlist = new Playlist(sfwStreams, { public: true })
} const filepath = 'index.m3u'
} await this.storage.save(filepath, playlist.toString())
this.logFile.append(
JSON.stringify({ type: 'index', filepath, count: playlist.streams.count() }) + EOL
)
}
}

111
scripts/generators/indexLanguageGenerator.ts
View File

@@ -1,54 +1,57 @@
import { Collection, Storage, File } from '@freearhey/core' import { Storage, File } from '@freearhey/storage-js'
import { Stream, Playlist, Language } from '../models' import { PUBLIC_DIR, EOL } from '../constants'
import { PUBLIC_DIR, EOL } from '../constants' import { Stream, Playlist } from '../models'
import { Generator } from './generator' import { Collection } from '@freearhey/core'
import { Generator } from './generator'
type IndexLanguageGeneratorProps = { import * as sdk from '@iptv-org/sdk'
streams: Collection
logFile: File type IndexLanguageGeneratorProps = {
} streams: Collection<Stream>
logFile: File
export class IndexLanguageGenerator implements Generator { }
streams: Collection
storage: Storage export class IndexLanguageGenerator implements Generator {
logFile: File streams: Collection<Stream>
storage: Storage
constructor({ streams, logFile }: IndexLanguageGeneratorProps) { logFile: File
this.streams = streams.clone()
this.storage = new Storage(PUBLIC_DIR) constructor({ streams, logFile }: IndexLanguageGeneratorProps) {
this.logFile = logFile this.streams = streams.clone()
} this.storage = new Storage(PUBLIC_DIR)
this.logFile = logFile
async generate(): Promise<void> { }
let groupedStreams = new Collection()
this.streams async generate(): Promise<void> {
.orderBy((stream: Stream) => stream.getTitle()) let groupedStreams = new Collection<Stream>()
.filter((stream: Stream) => stream.isSFW()) this.streams
.forEach((stream: Stream) => { .sortBy((stream: Stream) => stream.title)
if (!stream.hasLanguages()) { .filter((stream: Stream) => stream.isSFW())
const streamClone = stream.clone() .forEach((stream: Stream) => {
streamClone.groupTitle = 'Undefined' const streamLanguages = stream.getLanguages()
groupedStreams.add(streamClone) if (streamLanguages.isEmpty()) {
return const streamClone = stream.clone()
} streamClone.groupTitle = 'Undefined'
groupedStreams.add(streamClone)
stream.getLanguages().forEach((language: Language) => { return
const streamClone = stream.clone() }
streamClone.groupTitle = language.name
groupedStreams.add(streamClone) streamLanguages.forEach((language: sdk.Models.Language) => {
}) const streamClone = stream.clone()
}) streamClone.groupTitle = language.name
groupedStreams.add(streamClone)
groupedStreams = groupedStreams.orderBy((stream: Stream) => { })
if (stream.groupTitle === 'Undefined') return 'ZZ' })
return stream.groupTitle
}) groupedStreams = groupedStreams.sortBy((stream: Stream) => {
if (stream.groupTitle === 'Undefined') return 'ZZ'
const playlist = new Playlist(groupedStreams, { public: true }) return stream.groupTitle
const filepath = 'index.language.m3u' })
await this.storage.save(filepath, playlist.toString())
this.logFile.append( const playlist = new Playlist(groupedStreams, { public: true })
JSON.stringify({ type: 'index', filepath, count: playlist.streams.count() }) + EOL const filepath = 'index.language.m3u'
) await this.storage.save(filepath, playlist.toString())
} this.logFile.append(
} JSON.stringify({ type: 'index', filepath, count: playlist.streams.count() }) + EOL
)
}
}

32
scripts/generators/indexNsfwGenerator.ts
View File

@@ -1,32 +0,0 @@
import { Collection, File, Storage } from '@freearhey/core'
import { Stream, Playlist } from '../models'
import { PUBLIC_DIR, EOL } from '../constants'
import { Generator } from './generator'
type IndexNsfwGeneratorProps = {
streams: Collection
logFile: File
}
export class IndexNsfwGenerator implements Generator {
streams: Collection
storage: Storage
logFile: File
constructor({ streams, logFile }: IndexNsfwGeneratorProps) {
this.streams = streams.clone()
this.storage = new Storage(PUBLIC_DIR)
this.logFile = logFile
}
async generate(): Promise<void> {
const allStreams = this.streams.orderBy((stream: Stream) => stream.getTitle())
const playlist = new Playlist(allStreams, { public: true })
const filepath = 'index.nsfw.m3u'
await this.storage.save(filepath, playlist.toString())
this.logFile.append(
JSON.stringify({ type: 'index', filepath, count: playlist.streams.count() }) + EOL
)
}
}

115
scripts/generators/languagesGenerator.ts
View File

@@ -1,57 +1,58 @@
import { Collection, Storage, File } from '@freearhey/core' import { Storage, File } from '@freearhey/storage-js'
import { Playlist, Language, Stream } from '../models' import { PUBLIC_DIR, EOL } from '../constants'
import { PUBLIC_DIR, EOL } from '../constants' import { Playlist, Stream } from '../models'
import { Generator } from './generator' import { Collection } from '@freearhey/core'
import { Generator } from './generator'
type LanguagesGeneratorProps = { streams: Collection; logFile: File } import * as sdk from '@iptv-org/sdk'
export class LanguagesGenerator implements Generator { type LanguagesGeneratorProps = { streams: Collection<Stream>; logFile: File }
streams: Collection
storage: Storage export class LanguagesGenerator implements Generator {
logFile: File streams: Collection<Stream>
storage: Storage
constructor({ streams, logFile }: LanguagesGeneratorProps) { logFile: File
this.streams = streams.clone()
this.storage = new Storage(PUBLIC_DIR) constructor({ streams, logFile }: LanguagesGeneratorProps) {
this.logFile = logFile this.streams = streams.clone()
} this.storage = new Storage(PUBLIC_DIR)
this.logFile = logFile
async generate(): Promise<void> { }
const streams = this.streams
.orderBy((stream: Stream) => stream.getTitle()) async generate(): Promise<void> {
.filter((stream: Stream) => stream.isSFW()) const streams: Collection<Stream> = this.streams
.sortBy((stream: Stream) => stream.title)
let languages = new Collection() .filter((stream: Stream) => stream.isSFW())
streams.forEach((stream: Stream) => {
languages = languages.concat(stream.getLanguages()) const languages = new Collection<sdk.Models.Language>()
}) streams.forEach((stream: Stream) => {
languages.concat(stream.getLanguages())
languages })
.filter(Boolean)
.uniqBy((language: Language) => language.code) languages
.orderBy((language: Language) => language.name) .filter(Boolean)
.forEach(async (language: Language) => { .uniqBy((language: sdk.Models.Language) => language.code)
const languageStreams = streams.filter((stream: Stream) => stream.hasLanguage(language)) .sortBy((language: sdk.Models.Language) => language.name)
.forEach(async (language: sdk.Models.Language) => {
if (languageStreams.isEmpty()) return const languageStreams = streams.filter((stream: Stream) => stream.hasLanguage(language))
const playlist = new Playlist(languageStreams, { public: true }) if (languageStreams.isEmpty()) return
const filepath = `languages/${language.code}.m3u`
await this.storage.save(filepath, playlist.toString()) const playlist = new Playlist(languageStreams, { public: true })
this.logFile.append( const filepath = `languages/${language.code}.m3u`
JSON.stringify({ type: 'language', filepath, count: playlist.streams.count() }) + EOL await this.storage.save(filepath, playlist.toString())
) this.logFile.append(
}) JSON.stringify({ type: 'language', filepath, count: playlist.streams.count() }) + EOL
)
const undefinedStreams = streams.filter((stream: Stream) => !stream.hasLanguages()) })
if (undefinedStreams.isEmpty()) return const undefinedStreams = streams.filter((stream: Stream) => stream.getLanguages().isEmpty())
if (undefinedStreams.isEmpty()) return
const playlist = new Playlist(undefinedStreams, { public: true })
const filepath = 'languages/undefined.m3u' const playlist = new Playlist(undefinedStreams, { public: true })
await this.storage.save(filepath, playlist.toString()) const filepath = 'languages/undefined.m3u'
this.logFile.append( await this.storage.save(filepath, playlist.toString())
JSON.stringify({ type: 'language', filepath, count: playlist.streams.count() }) + EOL this.logFile.append(
) JSON.stringify({ type: 'language', filepath, count: playlist.streams.count() }) + EOL
} )
} }
}

85
scripts/generators/rawGenerator.ts
View File

@@ -1,40 +1,45 @@
import { Collection, Storage, File } from '@freearhey/core' import { Storage, File } from '@freearhey/storage-js'
import { Stream, Playlist } from '../models' import { PUBLIC_DIR, EOL } from '../constants'
import { PUBLIC_DIR, EOL } from '../constants' import { Stream, Playlist } from '../models'
import { Generator } from './generator' import { Collection } from '@freearhey/core'
import { Generator } from './generator'
type RawGeneratorProps = {
streams: Collection type RawGeneratorProps = {
logFile: File streams: Collection<Stream>
} logFile: File
}
export class RawGenerator implements Generator {
streams: Collection export class RawGenerator implements Generator {
storage: Storage streams: Collection<Stream>
logFile: File storage: Storage
logFile: File
constructor({ streams, logFile }: RawGeneratorProps) {
this.streams = streams.clone() constructor({ streams, logFile }: RawGeneratorProps) {
this.storage = new Storage(PUBLIC_DIR) this.streams = streams.clone()
this.logFile = logFile this.storage = new Storage(PUBLIC_DIR)
} this.logFile = logFile
}
async generate() {
const files = this.streams.groupBy((stream: Stream) => stream.getFilename()) async generate() {
const files = this.streams.groupBy((stream: Stream) => stream.getFilename())
for (const filename of files.keys()) {
const streams = new Collection(files.get(filename)).map((stream: Stream) => { for (const filename of files.keys()) {
const groupTitle = stream.getCategoryNames().join(';') const streams = new Collection(files.get(filename)).map((stream: Stream) => {
if (groupTitle) stream.groupTitle = groupTitle const groupTitle = stream
.getCategories()
return stream .map(category => category.name)
}) .sort()
const playlist = new Playlist(streams, { public: true }) .join(';')
const filepath = `raw/${filename}` if (groupTitle) stream.groupTitle = groupTitle
await this.storage.save(filepath, playlist.toString())
this.logFile.append( return stream
JSON.stringify({ type: 'raw', filepath, count: playlist.streams.count() }) + EOL })
) const playlist = new Playlist(streams, { public: true })
} const filepath = `raw/${filename}`
} await this.storage.save(filepath, playlist.toString())
} this.logFile.append(
JSON.stringify({ type: 'raw', filepath, count: playlist.streams.count() }) + EOL
)
}
}
}

95
scripts/generators/regionsGenerator.ts
View File

@@ -1,41 +1,54 @@
import { Collection, Storage, File } from '@freearhey/core' import { Storage, File } from '@freearhey/storage-js'
import { Playlist, Region, Stream } from '../models' import { PUBLIC_DIR, EOL } from '../constants'
import { PUBLIC_DIR, EOL } from '../constants' import { Playlist, Stream } from '../models'
import { Generator } from './generator' import { Collection } from '@freearhey/core'
import { Generator } from './generator'
type RegionsGeneratorProps = { import * as sdk from '@iptv-org/sdk'
streams: Collection
regions: Collection type RegionsGeneratorProps = {
logFile: File streams: Collection<Stream>
} regions: Collection<sdk.Models.Region>
logFile: File
export class RegionsGenerator implements Generator { }
streams: Collection
regions: Collection export class RegionsGenerator implements Generator {
storage: Storage streams: Collection<Stream>
logFile: File regions: Collection<sdk.Models.Region>
storage: Storage
constructor({ streams, regions, logFile }: RegionsGeneratorProps) { logFile: File
this.streams = streams.clone()
this.regions = regions constructor({ streams, regions, logFile }: RegionsGeneratorProps) {
this.storage = new Storage(PUBLIC_DIR) this.streams = streams.clone()
this.logFile = logFile this.regions = regions
} this.storage = new Storage(PUBLIC_DIR)
this.logFile = logFile
async generate(): Promise<void> { }
const streams = this.streams
.orderBy((stream: Stream) => stream.getTitle()) async generate(): Promise<void> {
.filter((stream: Stream) => stream.isSFW()) const streams = this.streams
.sortBy((stream: Stream) => stream.title)
this.regions.forEach(async (region: Region) => { .filter((stream: Stream) => stream.isSFW())
const regionStreams = streams.filter((stream: Stream) => stream.isBroadcastInRegion(region))
const streamsGroupedByRegionCode = {}
const playlist = new Playlist(regionStreams, { public: true }) streams.forEach((stream: Stream) => {
const filepath = `regions/${region.code.toLowerCase()}.m3u` stream.getBroadcastRegions().forEach((region: sdk.Models.Region) => {
await this.storage.save(filepath, playlist.toString()) if (streamsGroupedByRegionCode[region.code]) {
this.logFile.append( streamsGroupedByRegionCode[region.code].add(stream)
JSON.stringify({ type: 'region', filepath, count: playlist.streams.count() }) + EOL } else {
) streamsGroupedByRegionCode[region.code] = new Collection<Stream>([stream])
}) }
} })
} })
for (const regionCode in streamsGroupedByRegionCode) {
const regionStreams = streamsGroupedByRegionCode[regionCode]
const playlist = new Playlist(regionStreams, { public: true })
const filepath = `regions/${regionCode.toLowerCase()}.m3u`
await this.storage.save(filepath, playlist.toString())
this.logFile.append(
JSON.stringify({ type: 'region', filepath, count: playlist.streams.count() }) + EOL
)
}
}
}

92
scripts/generators/sourcesGenerator.ts
View File

@@ -1,43 +1,49 @@
import { Collection, Storage, File, type Dictionary } from '@freearhey/core' import { Collection, Dictionary } from '@freearhey/core'
import { Stream, Playlist } from '../models' import { Storage, File } from '@freearhey/storage-js'
import { PUBLIC_DIR, EOL } from '../constants' import { PUBLIC_DIR, EOL } from '../constants'
import { Generator } from './generator' import { Stream, Playlist } from '../models'
import { Generator } from './generator'
type SourcesGeneratorProps = {
streams: Collection type SourcesGeneratorProps = {
logFile: File streams: Collection<Stream>
} logFile: File
}
export class SourcesGenerator implements Generator {
streams: Collection export class SourcesGenerator implements Generator {
storage: Storage streams: Collection<Stream>
logFile: File storage: Storage
logFile: File
constructor({ streams, logFile }: SourcesGeneratorProps) {
this.streams = streams.clone() constructor({ streams, logFile }: SourcesGeneratorProps) {
this.storage = new Storage(PUBLIC_DIR) this.streams = streams.clone()
this.logFile = logFile this.storage = new Storage(PUBLIC_DIR)
} this.logFile = logFile
}
async generate() {
const files: Dictionary = this.streams.groupBy((stream: Stream) => stream.getFilename()) async generate() {
const files: Dictionary<Stream[]> = this.streams.groupBy((stream: Stream) =>
for (const filename of files.keys()) { stream.getFilename()
if (!filename) continue )
let streams = new Collection(files.get(filename)) for (const filename of files.keys()) {
streams = streams.map((stream: Stream) => { if (!filename) continue
const groupTitle = stream.getCategoryNames().join(';')
if (groupTitle) stream.groupTitle = groupTitle const streams = new Collection<Stream>(files.get(filename)).map((stream: Stream) => {
const groupTitle = stream
return stream .getCategories()
}) .map(category => category.name)
const playlist = new Playlist(streams, { public: true }) .sort()
const filepath = `sources/${filename}` .join(';')
await this.storage.save(filepath, playlist.toString()) if (groupTitle) stream.groupTitle = groupTitle
this.logFile.append(
JSON.stringify({ type: 'source', filepath, count: playlist.streams.count() }) + EOL return stream
) })
} const playlist = new Playlist(streams, { public: true })
} const filepath = `sources/${filename}`
} await this.storage.save(filepath, playlist.toString())
this.logFile.append(
JSON.stringify({ type: 'source', filepath, count: playlist.streams.count() }) + EOL
)
}
}
}

99
scripts/generators/subdivisionsGenerator.ts
View File

@@ -1,45 +1,54 @@
import { Subdivision, Stream, Playlist } from '../models' import { Storage, File } from '@freearhey/storage-js'
import { Collection, Storage, File } from '@freearhey/core' import { PUBLIC_DIR, EOL } from '../constants'
import { PUBLIC_DIR, EOL } from '../constants' import { Stream, Playlist } from '../models'
import { Generator } from './generator' import { Collection } from '@freearhey/core'
import { Generator } from './generator'
type SubdivisionsGeneratorProps = { import * as sdk from '@iptv-org/sdk'
streams: Collection
subdivisions: Collection type SubdivisionsGeneratorProps = {
logFile: File streams: Collection<Stream>
} subdivisions: Collection<sdk.Models.Subdivision>
logFile: File
export class SubdivisionsGenerator implements Generator { }
streams: Collection
subdivisions: Collection export class SubdivisionsGenerator implements Generator {
storage: Storage streams: Collection<Stream>
logFile: File subdivisions: Collection<sdk.Models.Subdivision>
storage: Storage
constructor({ streams, subdivisions, logFile }: SubdivisionsGeneratorProps) { logFile: File
this.streams = streams.clone()
this.subdivisions = subdivisions constructor({ streams, subdivisions, logFile }: SubdivisionsGeneratorProps) {
this.storage = new Storage(PUBLIC_DIR) this.streams = streams.clone()
this.logFile = logFile this.subdivisions = subdivisions
} this.storage = new Storage(PUBLIC_DIR)
this.logFile = logFile
async generate(): Promise<void> { }
const streams = this.streams
.orderBy((stream: Stream) => stream.getTitle()) async generate(): Promise<void> {
.filter((stream: Stream) => stream.isSFW()) const streams = this.streams
.sortBy((stream: Stream) => stream.title)
this.subdivisions.forEach(async (subdivision: Subdivision) => { .filter((stream: Stream) => stream.isSFW())
const subdivisionStreams = streams.filter((stream: Stream) =>
stream.isBroadcastInSubdivision(subdivision) const streamsGroupedBySubdivisionCode = {}
) streams.forEach((stream: Stream) => {
stream.getBroadcastSubdivisions().forEach((subdivision: sdk.Models.Subdivision) => {
if (subdivisionStreams.isEmpty()) return if (streamsGroupedBySubdivisionCode[subdivision.code]) {
streamsGroupedBySubdivisionCode[subdivision.code].add(stream)
const playlist = new Playlist(subdivisionStreams, { public: true }) } else {
const filepath = `subdivisions/${subdivision.code.toLowerCase()}.m3u` streamsGroupedBySubdivisionCode[subdivision.code] = new Collection<Stream>([stream])
await this.storage.save(filepath, playlist.toString()) }
this.logFile.append( })
JSON.stringify({ type: 'subdivision', filepath, count: playlist.streams.count() }) + EOL })
)
}) for (const subdivisionCode in streamsGroupedBySubdivisionCode) {
} const subdivisionStreams = streamsGroupedBySubdivisionCode[subdivisionCode]
}
const playlist = new Playlist(subdivisionStreams, { public: true })
const filepath = `subdivisions/${subdivisionCode.toLowerCase()}.m3u`
await this.storage.save(filepath, playlist.toString())
this.logFile.append(
JSON.stringify({ type: 'subdivision', filepath, count: playlist.streams.count() }) + EOL
)
}
}
}

15
scripts/models/blocklistRecord.ts
View File

@@ -1,15 +0,0 @@
import type { BlocklistRecordData } from '../types/blocklistRecord'
export class BlocklistRecord {
channelId: string
reason: string
ref: string
constructor(data?: BlocklistRecordData) {
if (!data) return
this.channelId = data.channel
this.reason = data.reason
this.ref = data.ref
}
}

108
scripts/models/broadcastArea.ts
View File

@@ -1,108 +0,0 @@
import { Collection, Dictionary } from '@freearhey/core'
import { City, Subdivision, Region, Country } from './'
export class BroadcastArea {
codes: Collection
citiesIncluded: Collection
subdivisionsIncluded: Collection
countriesIncluded: Collection
regionsIncluded: Collection
constructor(codes: Collection) {
this.codes = codes
}
withLocations(
citiesKeyByCode: Dictionary,
subdivisionsKeyByCode: Dictionary,
countriesKeyByCode: Dictionary,
regionsKeyByCode: Dictionary
): this {
const citiesIncluded = new Collection()
const subdivisionsIncluded = new Collection()
const countriesIncluded = new Collection()
let regionsIncluded = new Collection()
this.codes.forEach((value: string) => {
const [type, code] = value.split('/')
switch (type) {
case 'ct': {
const city: City = citiesKeyByCode.get(code)
if (!city) return
citiesIncluded.add(city)
if (city.subdivision) subdivisionsIncluded.add(city.subdivision)
if (city.subdivision && city.subdivision.parent)
subdivisionsIncluded.add(city.subdivision.parent)
if (city.country) countriesIncluded.add(city.country)
regionsIncluded = regionsIncluded.concat(city.getRegions())
break
}
case 's': {
const subdivision: Subdivision = subdivisionsKeyByCode.get(code)
if (!subdivision) return
subdivisionsIncluded.add(subdivision)
if (subdivision.country) countriesIncluded.add(subdivision.country)
regionsIncluded = regionsIncluded.concat(subdivision.getRegions())
break
}
case 'c': {
const country: Country = countriesKeyByCode.get(code)
if (!country) return
countriesIncluded.add(country)
regionsIncluded = regionsIncluded.concat(country.getRegions())
break
}
case 'r': {
const region: Region = regionsKeyByCode.get(code)
if (!region) return
regionsIncluded = regionsIncluded.concat(region.getRegions())
break
}
}
})
this.citiesIncluded = citiesIncluded.uniqBy((city: City) => city.code)
this.subdivisionsIncluded = subdivisionsIncluded.uniqBy(
(subdivision: Subdivision) => subdivision.code
)
this.countriesIncluded = countriesIncluded.uniqBy((country: Country) => country.code)
this.regionsIncluded = regionsIncluded.uniqBy((region: Region) => region.code)
return this
}
getCountries(): Collection {
return this.countriesIncluded || new Collection()
}
getSubdivisions(): Collection {
return this.subdivisionsIncluded || new Collection()
}
getCities(): Collection {
return this.citiesIncluded || new Collection()
}
getRegions(): Collection {
return this.regionsIncluded || new Collection()
}
includesCountry(country: Country): boolean {
return this.getCountries().includes((_country: Country) => _country.code === country.code)
}
includesSubdivision(subdivision: Subdivision): boolean {
return this.getSubdivisions().includes(
(_subdivision: Subdivision) => _subdivision.code === subdivision.code
)
}
includesRegion(region: Region): boolean {
return this.getRegions().includes((_region: Region) => _region.code === region.code)
}
includesCity(city: City): boolean {
return this.getCities().includes((_city: City) => _city.code === city.code)
}
}

18
scripts/models/category.ts
View File

@@ -1,18 +0,0 @@
import type { CategoryData, CategorySerializedData } from '../types/category'
export class Category {
id: string
name: string
constructor(data: CategoryData) {
this.id = data.id
this.name = data.name
}
serialize(): CategorySerializedData {
return {
id: this.id,
name: this.name
}
}
}

233
scripts/models/channel.ts
View File

@@ -1,233 +0,0 @@
import { Collection, Dictionary } from '@freearhey/core'
import { Category, Country, Feed, Guide, Logo, Stream, Subdivision } from './index'
import type { ChannelData, ChannelSearchableData, ChannelSerializedData } from '../types/channel'
export class Channel {
id: string
name: string
altNames: Collection
network?: string
owners: Collection
countryCode: string
country?: Country
subdivisionCode?: string
subdivision?: Subdivision
cityName?: string
categoryIds: Collection
categories: Collection = new Collection()
isNSFW: boolean
launched?: string
closed?: string
replacedBy?: string
isClosed: boolean
website?: string
feeds?: Collection
logos: Collection = new Collection()
constructor(data?: ChannelData) {
if (!data) return
this.id = data.id
this.name = data.name
this.altNames = new Collection(data.alt_names)
this.network = data.network || undefined
this.owners = new Collection(data.owners)
this.countryCode = data.country
this.subdivisionCode = data.subdivision || undefined
this.cityName = data.city || undefined
this.categoryIds = new Collection(data.categories)
this.isNSFW = data.is_nsfw
this.launched = data.launched || undefined
this.closed = data.closed || undefined
this.replacedBy = data.replaced_by || undefined
this.website = data.website || undefined
this.isClosed = !!data.closed || !!data.replaced_by
}
withSubdivision(subdivisionsKeyByCode: Dictionary): this {
if (!this.subdivisionCode) return this
this.subdivision = subdivisionsKeyByCode.get(this.subdivisionCode)
return this
}
withCountry(countriesKeyByCode: Dictionary): this {
this.country = countriesKeyByCode.get(this.countryCode)
return this
}
withCategories(categoriesKeyById: Dictionary): this {
this.categories = this.categoryIds
.map((id: string) => categoriesKeyById.get(id))
.filter(Boolean)
return this
}
withFeeds(feedsGroupedByChannelId: Dictionary): this {
this.feeds = new Collection(feedsGroupedByChannelId.get(this.id))
return this
}
withLogos(logosGroupedByChannelId: Dictionary): this {
if (this.id) this.logos = new Collection(logosGroupedByChannelId.get(this.id))
return this
}
getCountry(): Country | undefined {
return this.country
}
getSubdivision(): Subdivision | undefined {
return this.subdivision
}
getCategories(): Collection {
return this.categories || new Collection()
}
hasCategories(): boolean {
return !!this.categories && this.categories.notEmpty()
}
hasCategory(category: Category): boolean {
return (
!!this.categories &&
this.categories.includes((_category: Category) => _category.id === category.id)
)
}
getFeeds(): Collection {
if (!this.feeds) return new Collection()
return this.feeds
}
getGuides(): Collection {
let guides = new Collection()
this.getFeeds().forEach((feed: Feed) => {
guides = guides.concat(feed.getGuides())
})
return guides
}
getGuideNames(): Collection {
return this.getGuides()
.map((guide: Guide) => guide.siteName)
.uniq()
}
getStreams(): Collection {
let streams = new Collection()
this.getFeeds().forEach((feed: Feed) => {
streams = streams.concat(feed.getStreams())
})
return streams
}
getStreamTitles(): Collection {
return this.getStreams()
.map((stream: Stream) => stream.getTitle())
.uniq()
}
getFeedFullNames(): Collection {
return this.getFeeds()
.map((feed: Feed) => feed.getFullName())
.uniq()
}
isSFW(): boolean {
return this.isNSFW === false
}
getLogos(): Collection {
function feed(logo: Logo): number {
if (!logo.feed) return 1
if (logo.feed.isMain) return 1
return 0
}
function format(logo: Logo): number {
const levelByFormat = { SVG: 0, PNG: 3, APNG: 1, WebP: 1, AVIF: 1, JPEG: 2, GIF: 1 }
return logo.format ? levelByFormat[logo.format] : 0
}
function size(logo: Logo): number {
return Math.abs(512 - logo.width) + Math.abs(512 - logo.height)
}
return this.logos.orderBy([feed, format, size], ['desc', 'desc', 'asc'], false)
}
getLogo(): Logo | undefined {
return this.getLogos().first()
}
hasLogo(): boolean {
return this.getLogos().notEmpty()
}
getSearchable(): ChannelSearchableData {
return {
id: this.id,
name: this.name,
altNames: this.altNames.all(),
guideNames: this.getGuideNames().all(),
streamTitles: this.getStreamTitles().all(),
feedFullNames: this.getFeedFullNames().all()
}
}
serialize(): ChannelSerializedData {
return {
id: this.id,
name: this.name,
altNames: this.altNames.all(),
network: this.network,
owners: this.owners.all(),
countryCode: this.countryCode,
country: this.country ? this.country.serialize() : undefined,
subdivisionCode: this.subdivisionCode,
subdivision: this.subdivision ? this.subdivision.serialize() : undefined,
cityName: this.cityName,
categoryIds: this.categoryIds.all(),
categories: this.categories.map((category: Category) => category.serialize()).all(),
isNSFW: this.isNSFW,
launched: this.launched,
closed: this.closed,
replacedBy: this.replacedBy,
website: this.website
}
}
deserialize(data: ChannelSerializedData): this {
this.id = data.id
this.name = data.name
this.altNames = new Collection(data.altNames)
this.network = data.network
this.owners = new Collection(data.owners)
this.countryCode = data.countryCode
this.country = data.country ? new Country().deserialize(data.country) : undefined
this.subdivisionCode = data.subdivisionCode
this.cityName = data.cityName
this.categoryIds = new Collection(data.categoryIds)
this.isNSFW = data.isNSFW
this.launched = data.launched
this.closed = data.closed
this.replacedBy = data.replacedBy
this.website = data.website
return this
}
}

78
scripts/models/city.ts
View File

@@ -1,78 +0,0 @@
import { Collection, Dictionary } from '@freearhey/core'
import { Country, Region, Subdivision } from '.'
import type { CityData, CitySerializedData } from '../types/city'
export class City {
code: string
name: string
countryCode: string
country?: Country
subdivisionCode?: string
subdivision?: Subdivision
wikidataId: string
regions?: Collection
constructor(data?: CityData) {
if (!data) return
this.code = data.code
this.name = data.name
this.countryCode = data.country
this.subdivisionCode = data.subdivision || undefined
this.wikidataId = data.wikidata_id
}
withCountry(countriesKeyByCode: Dictionary): this {
this.country = countriesKeyByCode.get(this.countryCode)
return this
}
withSubdivision(subdivisionsKeyByCode: Dictionary): this {
if (!this.subdivisionCode) return this
this.subdivision = subdivisionsKeyByCode.get(this.subdivisionCode)
return this
}
withRegions(regions: Collection): this {
this.regions = regions.filter((region: Region) =>
region.countryCodes.includes(this.countryCode)
)
return this
}
getRegions(): Collection {
if (!this.regions) return new Collection()
return this.regions
}
serialize(): CitySerializedData {
return {
code: this.code,
name: this.name,
countryCode: this.countryCode,
country: this.country ? this.country.serialize() : undefined,
subdivisionCode: this.subdivisionCode || null,
subdivision: this.subdivision ? this.subdivision.serialize() : undefined,
wikidataId: this.wikidataId
}
}
deserialize(data: CitySerializedData): this {
this.code = data.code
this.name = data.name
this.countryCode = data.countryCode
this.country = data.country ? new Country().deserialize(data.country) : undefined
this.subdivisionCode = data.subdivisionCode || undefined
this.subdivision = data.subdivision
? new Subdivision().deserialize(data.subdivision)
: undefined
this.wikidataId = data.wikidataId
return this
}
}

95
scripts/models/country.ts
View File

@@ -1,95 +0,0 @@
import { Collection, Dictionary } from '@freearhey/core'
import { Region, Language, Subdivision } from '.'
import type { CountryData, CountrySerializedData } from '../types/country'
import { SubdivisionSerializedData } from '../types/subdivision'
import { RegionSerializedData } from '../types/region'
export class Country {
code: string
name: string
flag: string
languageCode: string
language?: Language
subdivisions?: Collection
regions?: Collection
cities?: Collection
constructor(data?: CountryData) {
if (!data) return
this.code = data.code
this.name = data.name
this.flag = data.flag
this.languageCode = data.lang
}
withSubdivisions(subdivisionsGroupedByCountryCode: Dictionary): this {
this.subdivisions = new Collection(subdivisionsGroupedByCountryCode.get(this.code))
return this
}
withRegions(regions: Collection): this {
this.regions = regions.filter((region: Region) => region.includesCountryCode(this.code))
return this
}
withCities(citiesGroupedByCountryCode: Dictionary): this {
this.cities = new Collection(citiesGroupedByCountryCode.get(this.code))
return this
}
withLanguage(languagesKeyByCode: Dictionary): this {
this.language = languagesKeyByCode.get(this.languageCode)
return this
}
getLanguage(): Language | undefined {
return this.language
}
getRegions(): Collection {
return this.regions || new Collection()
}
getSubdivisions(): Collection {
return this.subdivisions || new Collection()
}
getCities(): Collection {
return this.cities || new Collection()
}
serialize(): CountrySerializedData {
return {
code: this.code,
name: this.name,
flag: this.flag,
languageCode: this.languageCode,
language: this.language ? this.language.serialize() : null,
subdivisions: this.subdivisions
? this.subdivisions.map((subdivision: Subdivision) => subdivision.serialize()).all()
: [],
regions: this.regions ? this.regions.map((region: Region) => region.serialize()).all() : []
}
}
deserialize(data: CountrySerializedData): this {
this.code = data.code
this.name = data.name
this.flag = data.flag
this.languageCode = data.languageCode
this.language = data.language ? new Language().deserialize(data.language) : undefined
this.subdivisions = new Collection(data.subdivisions).map((data: SubdivisionSerializedData) =>
new Subdivision().deserialize(data)
)
this.regions = new Collection(data.regions).map((data: RegionSerializedData) =>
new Region().deserialize(data)
)
return this
}
}

170
scripts/models/feed.ts
View File

@@ -1,170 +0,0 @@
import { Country, Language, Region, Channel, Subdivision, BroadcastArea, City } from './index'
import { Collection, Dictionary } from '@freearhey/core'
import type { FeedData } from '../types/feed'
export class Feed {
channelId: string
channel?: Channel
id: string
name: string
isMain: boolean
broadcastAreaCodes: Collection
broadcastArea?: BroadcastArea
languageCodes: Collection
languages?: Collection
timezoneIds: Collection
timezones?: Collection
videoFormat: string
guides?: Collection
streams?: Collection
constructor(data: FeedData) {
this.channelId = data.channel
this.id = data.id
this.name = data.name
this.isMain = data.is_main
this.broadcastAreaCodes = new Collection(data.broadcast_area)
this.languageCodes = new Collection(data.languages)
this.timezoneIds = new Collection(data.timezones)
this.videoFormat = data.video_format
}
withChannel(channelsKeyById: Dictionary): this {
this.channel = channelsKeyById.get(this.channelId)
return this
}
withStreams(streamsGroupedById: Dictionary): this {
this.streams = new Collection(streamsGroupedById.get(`${this.channelId}@${this.id}`))
if (this.isMain) {
this.streams = this.streams.concat(new Collection(streamsGroupedById.get(this.channelId)))
}
return this
}
withGuides(guidesGroupedByStreamId: Dictionary): this {
this.guides = new Collection(guidesGroupedByStreamId.get(`${this.channelId}@${this.id}`))
if (this.isMain) {
this.guides = this.guides.concat(new Collection(guidesGroupedByStreamId.get(this.channelId)))
}
return this
}
withLanguages(languagesKeyByCode: Dictionary): this {
this.languages = this.languageCodes
.map((code: string) => languagesKeyByCode.get(code))
.filter(Boolean)
return this
}
withTimezones(timezonesKeyById: Dictionary): this {
this.timezones = this.timezoneIds.map((id: string) => timezonesKeyById.get(id)).filter(Boolean)
return this
}
withBroadcastArea(
citiesKeyByCode: Dictionary,
subdivisionsKeyByCode: Dictionary,
countriesKeyByCode: Dictionary,
regionsKeyByCode: Dictionary
): this {
this.broadcastArea = new BroadcastArea(this.broadcastAreaCodes).withLocations(
citiesKeyByCode,
subdivisionsKeyByCode,
countriesKeyByCode,
regionsKeyByCode
)
return this
}
hasBroadcastArea(): boolean {
return !!this.broadcastArea
}
getBroadcastCountries(): Collection {
if (!this.broadcastArea) return new Collection()
return this.broadcastArea.getCountries()
}
getBroadcastRegions(): Collection {
if (!this.broadcastArea) return new Collection()
return this.broadcastArea.getRegions()
}
getTimezones(): Collection {
return this.timezones || new Collection()
}
getLanguages(): Collection {
return this.languages || new Collection()
}
hasLanguages(): boolean {
return !!this.languages && this.languages.notEmpty()
}
hasLanguage(language: Language): boolean {
return (
!!this.languages &&
this.languages.includes((_language: Language) => _language.code === language.code)
)
}
isBroadcastInCity(city: City): boolean {
if (!this.broadcastArea) return false
return this.broadcastArea.includesCity(city)
}
isBroadcastInSubdivision(subdivision: Subdivision): boolean {
if (!this.broadcastArea) return false
return this.broadcastArea.includesSubdivision(subdivision)
}
isBroadcastInCountry(country: Country): boolean {
if (!this.broadcastArea) return false
return this.broadcastArea.includesCountry(country)
}
isBroadcastInRegion(region: Region): boolean {
if (!this.broadcastArea) return false
return this.broadcastArea.includesRegion(region)
}
isInternational(): boolean {
if (!this.broadcastArea) return false
return this.broadcastArea.codes.join(',').includes('r/')
}
getGuides(): Collection {
if (!this.guides) return new Collection()
return this.guides
}
getStreams(): Collection {
if (!this.streams) return new Collection()
return this.streams
}
getFullName(): string {
if (!this.channel) return ''
return `${this.channel.name} ${this.name}`
}
}

54
scripts/models/guide.ts
View File

@@ -1,54 +0,0 @@
import type { GuideData, GuideSerializedData } from '../types/guide'
export class Guide {
channelId?: string
feedId?: string
siteDomain: string
siteId: string
siteName: string
languageCode: string
constructor(data?: GuideData) {
if (!data) return
this.channelId = data.channel
this.feedId = data.feed
this.siteDomain = data.site
this.siteId = data.site_id
this.siteName = data.site_name
this.languageCode = data.lang
}
getUUID(): string {
return this.getStreamId() + this.siteId
}
getStreamId(): string | undefined {
if (!this.channelId) return undefined
if (!this.feedId) return this.channelId
return `${this.channelId}@${this.feedId}`
}
serialize(): GuideSerializedData {
return {
channelId: this.channelId,
feedId: this.feedId,
siteDomain: this.siteDomain,
siteId: this.siteId,
siteName: this.siteName,
languageCode: this.languageCode
}
}
deserialize(data: GuideSerializedData): this {
this.channelId = data.channelId
this.feedId = data.feedId
this.siteDomain = data.siteDomain
this.siteId = data.siteId
this.siteName = data.siteName
this.languageCode = data.languageCode
return this
}
}

19
scripts/models/index.ts
View File

@@ -1,16 +1,3 @@
export * from './blocklistRecord' export * from './issue'
export * from './broadcastArea' export * from './playlist'
export * from './category' export * from './stream'
export * from './channel'
export * from './city'
export * from './country'
export * from './feed'
export * from './guide'
export * from './issue'
export * from './language'
export * from './logo'
export * from './playlist'
export * from './region'
export * from './stream'
export * from './subdivision'
export * from './timezone'

38
scripts/models/issue.ts
View File

@@ -1,19 +1,19 @@
import { IssueData } from '../core' import { IssueData } from '../core'
type IssueProps = { type IssueProps = {
number: number number: number
labels: string[] labels: string[]
data: IssueData data: IssueData
} }
export class Issue { export class Issue {
number: number number: number
labels: string[] labels: string[]
data: IssueData data: IssueData
constructor({ number, labels, data }: IssueProps) { constructor({ number, labels, data }: IssueProps) {
this.number = number this.number = number
this.labels = labels this.labels = labels
this.data = data this.data = data
} }
} }

27
scripts/models/language.ts
View File

@@ -1,27 +0,0 @@
import type { LanguageData, LanguageSerializedData } from '../types/language'
export class Language {
code: string
name: string
constructor(data?: LanguageData) {
if (!data) return
this.code = data.code
this.name = data.name
}
serialize(): LanguageSerializedData {
return {
code: this.code,
name: this.name
}
}
deserialize(data: LanguageSerializedData): this {
this.code = data.code
this.name = data.name
return this
}
}

40
scripts/models/logo.ts
View File

@@ -1,40 +0,0 @@
import { Collection, type Dictionary } from '@freearhey/core'
import type { LogoData } from '../types/logo'
import { type Feed } from './feed'
export class Logo {
channelId: string
feedId?: string
feed: Feed
tags: Collection
width: number
height: number
format?: string
url: string
constructor(data?: LogoData) {
if (!data) return
this.channelId = data.channel
this.feedId = data.feed || undefined
this.tags = new Collection(data.tags)
this.width = data.width
this.height = data.height
this.format = data.format || undefined
this.url = data.url
}
withFeed(feedsKeyById: Dictionary): this {
if (!this.feedId) return this
this.feed = feedsKeyById.get(this.feedId)
return this
}
getStreamId(): string {
if (!this.feedId) return this.channelId
return `${this.channelId}@${this.feedId}`
}
}

56
scripts/models/playlist.ts
View File

@@ -1,28 +1,28 @@
import { Collection } from '@freearhey/core' import { Collection } from '@freearhey/core'
import { Stream } from '../models' import { Stream } from '../models'
type PlaylistOptions = { type PlaylistOptions = {
public: boolean public: boolean
} }
export class Playlist { export class Playlist {
streams: Collection streams: Collection<Stream>
options: { options: {
public: boolean public: boolean
} }
constructor(streams: Collection, options?: PlaylistOptions) { constructor(streams: Collection<Stream>, options?: PlaylistOptions) {
this.streams = streams this.streams = streams
this.options = options || { public: false } this.options = options || { public: false }
} }
toString() { toString() {
let output = '#EXTM3U\r\n' let output = '#EXTM3U\r\n'
this.streams.forEach((stream: Stream) => { this.streams.forEach((stream: Stream) => {
output += stream.toString(this.options) + '\r\n' output += stream.toString(this.options) + '\r\n'
}) })
return output return output
} }
} }

118
scripts/models/region.ts
View File

@@ -1,118 +0,0 @@
import { Collection, Dictionary } from '@freearhey/core'
import { City, Country, Subdivision } from '.'
import type { RegionData, RegionSerializedData } from '../types/region'
import { CountrySerializedData } from '../types/country'
import { SubdivisionSerializedData } from '../types/subdivision'
import { CitySerializedData } from '../types/city'
export class Region {
code: string
name: string
countryCodes: Collection
countries?: Collection
subdivisions?: Collection
cities?: Collection
regions?: Collection
constructor(data?: RegionData) {
if (!data) return
this.code = data.code
this.name = data.name
this.countryCodes = new Collection(data.countries)
}
withCountries(countriesKeyByCode: Dictionary): this {
this.countries = this.countryCodes.map((code: string) => countriesKeyByCode.get(code))
return this
}
withSubdivisions(subdivisions: Collection): this {
this.subdivisions = subdivisions.filter(
(subdivision: Subdivision) => this.countryCodes.indexOf(subdivision.countryCode) > -1
)
return this
}
withCities(cities: Collection): this {
this.cities = cities.filter((city: City) => this.countryCodes.indexOf(city.countryCode) > -1)
return this
}
withRegions(regions: Collection): this {
this.regions = regions.filter(
(region: Region) => !region.countryCodes.intersects(this.countryCodes).isEmpty()
)
return this
}
getSubdivisions(): Collection {
if (!this.subdivisions) return new Collection()
return this.subdivisions
}
getCountries(): Collection {
if (!this.countries) return new Collection()
return this.countries
}
getCities(): Collection {
if (!this.cities) return new Collection()
return this.cities
}
getRegions(): Collection {
if (!this.regions) return new Collection()
return this.regions
}
includesCountryCode(code: string): boolean {
return this.countryCodes.includes((countryCode: string) => countryCode === code)
}
isWorldwide(): boolean {
return ['INT', 'WW'].includes(this.code)
}
serialize(): RegionSerializedData {
return {
code: this.code,
name: this.name,
countryCodes: this.countryCodes.all(),
countries: this.getCountries()
.map((country: Country) => country.serialize())
.all(),
subdivisions: this.getSubdivisions()
.map((subdivision: Subdivision) => subdivision.serialize())
.all(),
cities: this.getCities()
.map((city: City) => city.serialize())
.all()
}
}
deserialize(data: RegionSerializedData): this {
this.code = data.code
this.name = data.name
this.countryCodes = new Collection(data.countryCodes)
this.countries = new Collection(data.countries).map((data: CountrySerializedData) =>
new Country().deserialize(data)
)
this.subdivisions = new Collection(data.subdivisions).map((data: SubdivisionSerializedData) =>
new Subdivision().deserialize(data)
)
this.cities = new Collection(data.cities).map((data: CitySerializedData) =>
new City().deserialize(data)
)
return this
}
}

935
scripts/models/stream.ts
View File

@@ -1,474 +1,461 @@
import { import { Collection } from '@freearhey/core'
Feed, import parser from 'iptv-playlist-parser'
Channel, import { normalizeURL } from '../utils'
Category, import * as sdk from '@iptv-org/sdk'
Region, import { IssueData } from '../core'
Subdivision, import { data } from '../api'
Country, import path from 'node:path'
Language,
Logo, export class Stream extends sdk.Models.Stream {
City directives: Collection<string>
} from './index' filepath?: string
import { URL, Collection, Dictionary } from '@freearhey/core' line?: number
import type { StreamData } from '../types/stream' groupTitle: string = 'Undefined'
import parser from 'iptv-playlist-parser' removed: boolean = false
import { IssueData } from '../core' tvgId?: string
import path from 'node:path' label: string | null
export class Stream { updateWithIssue(issueData: IssueData): this {
title: string const data = {
url: string label: issueData.getString('label'),
id?: string quality: issueData.getString('quality'),
channelId?: string httpUserAgent: issueData.getString('httpUserAgent'),
channel?: Channel httpReferrer: issueData.getString('httpReferrer'),
feedId?: string newStreamUrl: issueData.getString('newStreamUrl'),
feed?: Feed directives: issueData.getArray('directives')
logos: Collection = new Collection() }
filepath?: string
line?: number if (data.label !== undefined) this.label = data.label
label?: string if (data.quality !== undefined) this.quality = data.quality
verticalResolution?: number if (data.httpUserAgent !== undefined) this.user_agent = data.httpUserAgent
isInterlaced?: boolean if (data.httpReferrer !== undefined) this.referrer = data.httpReferrer
referrer?: string if (data.newStreamUrl !== undefined) this.url = data.newStreamUrl
userAgent?: string if (data.directives !== undefined) this.setDirectives(data.directives)
groupTitle: string = 'Undefined'
removed: boolean = false return this
directives: Collection = new Collection() }
constructor(data?: StreamData) { static fromPlaylistItem(data: parser.PlaylistItem): Stream {
if (!data) return function escapeRegExp(text) {
return text.replace(/[-[\]{}()*+?.,\\^$|#\s]/g, '\\$&')
const id = }
data.channelId && data.feedId ? [data.channelId, data.feedId].join('@') : data.channelId
const { verticalResolution, isInterlaced } = parseQuality(data.quality) function parseName(name: string): {
title: string
this.id = id || undefined label: string
this.channelId = data.channelId || undefined quality: string
this.feedId = data.feedId || undefined } {
this.title = data.title || '' let title = name
this.url = data.url const [, label] = title.match(/ \[(.*)\]$/) || [null, '']
this.referrer = data.referrer || undefined title = title.replace(new RegExp(` \\[${escapeRegExp(label)}\\]$`), '')
this.userAgent = data.userAgent || undefined const [, quality] = title.match(/ \(([0-9]+[p|i])\)$/) || [null, '']
this.verticalResolution = verticalResolution || undefined title = title.replace(new RegExp(` \\(${quality}\\)$`), '')
this.isInterlaced = isInterlaced || undefined
this.label = data.label || undefined return { title, label, quality }
this.directives = new Collection(data.directives) }
}
function parseDirectives(string: string): Collection<string> {
update(issueData: IssueData): this { const directives = new Collection<string>()
const data = {
label: issueData.getString('label'), if (!string) return directives
quality: issueData.getString('quality'),
httpUserAgent: issueData.getString('httpUserAgent'), const supportedDirectives = ['#EXTVLCOPT', '#KODIPROP']
httpReferrer: issueData.getString('httpReferrer'), const lines = string.split('\r\n')
newStreamUrl: issueData.getString('newStreamUrl'), const regex = new RegExp(`^${supportedDirectives.join('|')}`, 'i')
directives: issueData.getArray('directives')
} lines.forEach((line: string) => {
if (regex.test(line)) {
if (data.label !== undefined) this.label = data.label directives.add(line.trim())
if (data.quality !== undefined) this.setQuality(data.quality) }
if (data.httpUserAgent !== undefined) this.userAgent = data.httpUserAgent })
if (data.httpReferrer !== undefined) this.referrer = data.httpReferrer
if (data.newStreamUrl !== undefined) this.url = data.newStreamUrl return directives
if (data.directives !== undefined) this.directives = new Collection(data.directives) }
return this if (!data.name) throw new Error('"name" property is required')
} if (!data.url) throw new Error('"url" property is required')
fromPlaylistItem(data: parser.PlaylistItem): this { const [channelId, feedId] = data.tvg.id.split('@')
function parseName(name: string): { const { title, label, quality } = parseName(data.name)
title: string
label: string const stream = new Stream({
quality: string channel: channelId || null,
} { feed: feedId || null,
let title = name title: title,
const [, label] = title.match(/ \[(.*)\]$/) || [null, ''] quality: quality || null,
title = title.replace(new RegExp(` \\[${escapeRegExp(label)}\\]$`), '') url: data.url,
const [, quality] = title.match(/ \(([0-9]+p)\)$/) || [null, ''] referrer: data.http.referrer || null,
title = title.replace(new RegExp(` \\(${quality}\\)$`), '') user_agent: data.http['user-agent'] || null
})
return { title, label, quality }
} stream.tvgId = data.tvg.id
stream.line = data.line
function parseDirectives(string: string) { stream.label = label || null
const directives = new Collection() stream.directives = parseDirectives(data.raw)
if (!string) return directives return stream
}
const supportedDirectives = ['#EXTVLCOPT', '#KODIPROP']
const lines = string.split('\r\n') isSFW(): boolean {
const regex = new RegExp(`^${supportedDirectives.join('|')}`, 'i') const channel = this.getChannel()
lines.forEach((line: string) => { if (!channel) return true
if (regex.test(line)) {
directives.add(line.trim()) return !channel.is_nsfw
} }
})
getUniqKey(): string {
return directives const filepath = this.getFilepath()
} const tvgId = this.getTvgId()
if (!data.name) throw new Error('"name" property is required') return filepath + tvgId + this.url
if (!data.url) throw new Error('"url" property is required') }
const [channelId, feedId] = data.tvg.id.split('@') getVerticalResolution(): number {
const { title, label, quality } = parseName(data.name) if (!this.quality) return 0
const { verticalResolution, isInterlaced } = parseQuality(quality)
const [, verticalResolutionString] = this.quality.match(/^(\d+)/) || ['', '0']
this.id = data.tvg.id || undefined
this.feedId = feedId || undefined return parseInt(verticalResolutionString)
this.channelId = channelId || undefined }
this.line = data.line
this.label = label || undefined getBroadcastCountries(): Collection<sdk.Models.Country> {
this.title = title const countries = new Collection<sdk.Models.Country>()
this.verticalResolution = verticalResolution || undefined
this.isInterlaced = isInterlaced || undefined const feed = this.getFeed()
this.url = data.url if (!feed) return countries
this.referrer = data.http.referrer || undefined
this.userAgent = data.http['user-agent'] || undefined feed
this.directives = parseDirectives(data.raw) .getBroadcastArea()
.getLocations()
return this .forEach((location: sdk.Models.BroadcastAreaLocation) => {
} let country: sdk.Models.Country | undefined
switch (location.type) {
withChannel(channelsKeyById: Dictionary): this { case 'country': {
if (!this.channelId) return this country = data.countriesKeyByCode.get(location.code)
break
this.channel = channelsKeyById.get(this.channelId) }
case 'subdivision': {
return this const subdivision = data.subdivisionsKeyByCode.get(location.code)
} if (!subdivision) break
country = data.countriesKeyByCode.get(subdivision.country)
withFeed(feedsGroupedByChannelId: Dictionary): this { break
if (!this.channelId) return this }
case 'city': {
const channelFeeds = feedsGroupedByChannelId.get(this.channelId) || [] const city = data.citiesKeyByCode.get(location.code)
if (this.feedId) this.feed = channelFeeds.find((feed: Feed) => feed.id === this.feedId) if (!city) break
if (!this.feedId && !this.feed) this.feed = channelFeeds.find((feed: Feed) => feed.isMain) country = data.countriesKeyByCode.get(city.country)
break
return this }
} }
withLogos(logosGroupedByStreamId: Dictionary): this { if (country) countries.add(country)
if (this.id) this.logos = new Collection(logosGroupedByStreamId.get(this.id)) })
return this return countries.uniqBy((country: sdk.Models.Country) => country.code)
} }
setId(id: string): this { getBroadcastSubdivisions(): Collection<sdk.Models.Subdivision> {
this.id = id const subdivisions = new Collection<sdk.Models.Subdivision>()
return this const feed = this.getFeed()
} if (!feed) return subdivisions
setChannelId(channelId: string): this { feed
this.channelId = channelId .getBroadcastArea()
.getLocations()
return this .forEach((location: sdk.Models.BroadcastAreaLocation) => {
} switch (location.type) {
case 'subdivision': {
setFeedId(feedId: string | undefined): this { const subdivision = data.subdivisionsKeyByCode.get(location.code)
this.feedId = feedId if (!subdivision) break
subdivisions.add(subdivision)
return this if (!subdivision.parent) break
} const parentSubdivision = data.subdivisionsKeyByCode.get(subdivision.parent)
if (!parentSubdivision) break
setQuality(quality: string): this { subdivisions.add(parentSubdivision)
const { verticalResolution, isInterlaced } = parseQuality(quality) break
}
this.verticalResolution = verticalResolution || undefined case 'city': {
this.isInterlaced = isInterlaced || undefined const city = data.citiesKeyByCode.get(location.code)
if (!city || !city.subdivision) break
return this const subdivision = data.subdivisionsKeyByCode.get(city.subdivision)
} if (!subdivision) break
subdivisions.add(subdivision)
getLine(): number { if (!subdivision.parent) break
return this.line || -1 const parentSubdivision = data.subdivisionsKeyByCode.get(subdivision.parent)
} if (!parentSubdivision) break
subdivisions.add(parentSubdivision)
getFilename(): string { break
if (!this.filepath) return '' }
}
return path.basename(this.filepath) })
}
return subdivisions.uniqBy((subdivision: sdk.Models.Subdivision) => subdivision.code)
setFilepath(filepath: string): this { }
this.filepath = filepath
getBroadcastCities(): Collection<sdk.Models.City> {
return this const cities = new Collection<sdk.Models.City>()
}
const feed = this.getFeed()
updateFilepath(): this { if (!feed) return cities
if (!this.channel) return this
feed
this.filepath = `${this.channel.countryCode.toLowerCase()}.m3u` .getBroadcastArea()
.getLocations()
return this .forEach((location: sdk.Models.BroadcastAreaLocation) => {
} if (location.type !== 'city') return
getChannelId(): string { const city = data.citiesKeyByCode.get(location.code)
return this.channelId || ''
} if (city) cities.add(city)
})
getFeedId(): string {
if (this.feedId) return this.feedId return cities.uniqBy((city: sdk.Models.City) => city.code)
if (this.feed) return this.feed.id }
return ''
} getBroadcastRegions(): Collection<sdk.Models.Region> {
const regions = new Collection<sdk.Models.Region>()
getFilepath(): string {
return this.filepath || '' const feed = this.getFeed()
} if (!feed) return regions
getReferrer(): string { feed
return this.referrer || '' .getBroadcastArea()
} .getLocations()
.forEach((location: sdk.Models.BroadcastAreaLocation) => {
getUserAgent(): string { switch (location.type) {
return this.userAgent || '' case 'region': {
} const region = data.regionsKeyByCode.get(location.code)
if (!region) break
getQuality(): string { regions.add(region)
if (!this.verticalResolution) return ''
const relatedRegions = data.regions.filter((_region: sdk.Models.Region) =>
let quality = this.verticalResolution.toString() new Collection<string>(_region.countries)
.intersects(new Collection<string>(region.countries))
if (this.isInterlaced) quality += 'i' .isNotEmpty()
else quality += 'p' )
regions.concat(relatedRegions)
return quality break
} }
case 'country': {
hasId(): boolean { const country = data.countriesKeyByCode.get(location.code)
return !!this.id if (!country) break
} const countryRegions = data.regions.filter((_region: sdk.Models.Region) =>
new Collection<string>(_region.countries).includes(
hasQuality(): boolean { (code: string) => code === country.code
return !!this.verticalResolution )
} )
regions.concat(countryRegions)
getVerticalResolution(): number { break
if (!this.hasQuality()) return 0 }
case 'subdivision': {
return parseInt(this.getQuality().replace(/p|i/, '')) const subdivision = data.subdivisionsKeyByCode.get(location.code)
} if (!subdivision) break
const subdivisionRegions = data.regions.filter((_region: sdk.Models.Region) =>
updateTitle(): this { new Collection<string>(_region.countries).includes(
if (!this.channel) return this (code: string) => code === subdivision.country
)
this.title = this.channel.name )
if (this.feed && !this.feed.isMain) { regions.concat(subdivisionRegions)
this.title += ` ${this.feed.name}` break
} }
case 'city': {
return this const city = data.citiesKeyByCode.get(location.code)
} if (!city) break
const cityRegions = data.regions.filter((_region: sdk.Models.Region) =>
updateId(): this { new Collection<string>(_region.countries).includes(
if (!this.channel) return this (code: string) => code === city.country
if (this.feed) { )
this.id = `${this.channel.id}@${this.feed.id}` )
} else { regions.concat(cityRegions)
this.id = this.channel.id break
} }
}
return this })
}
return regions.uniqBy((region: sdk.Models.Region) => region.code)
normalizeURL() { }
const url = new URL(this.url)
isInternational(): boolean {
this.url = url.normalize().toString() const feed = this.getFeed()
} if (!feed) return false
clone(): Stream { const broadcastAreaCodes = feed.getBroadcastArea().codes
return Object.assign(Object.create(Object.getPrototypeOf(this)), this) if (broadcastAreaCodes.join(';').includes('r/')) return true
} if (broadcastAreaCodes.filter(code => code.includes('c/')).length > 1) return true
hasChannel() { return false
return !!this.channel }
}
hasCategory(category: sdk.Models.Category): boolean {
getBroadcastRegions(): Collection { const channel = this.getChannel()
return this.feed ? this.feed.getBroadcastRegions() : new Collection()
} if (!channel) return false
getBroadcastCountries(): Collection { const found = channel.categories.find((id: string) => id === category.id)
return this.feed ? this.feed.getBroadcastCountries() : new Collection()
} return !!found
}
hasBroadcastArea(): boolean {
return this.feed ? this.feed.hasBroadcastArea() : false hasLanguage(language: sdk.Models.Language): boolean {
} const found = this.getLanguages().find(
(_language: sdk.Models.Language) => _language.code === language.code
isSFW(): boolean { )
return this.channel ? this.channel.isSFW() : true
} return !!found
}
hasCategories(): boolean {
return this.channel ? this.channel.hasCategories() : false setDirectives(directives: string[]): this {
} this.directives = new Collection(directives).filter((directive: string) =>
/^(#KODIPROP|#EXTVLCOPT)/.test(directive)
hasCategory(category: Category): boolean { )
return this.channel ? this.channel.hasCategory(category) : false
} return this
}
getCategoryNames(): string[] {
return this.getCategories() updateTvgId(): this {
.map((category: Category) => category.name) if (!this.channel) return this
.sort() if (this.feed) {
.all() this.tvgId = `${this.channel}@${this.feed}`
} } else {
this.tvgId = this.channel
getCategories(): Collection { }
return this.channel ? this.channel.getCategories() : new Collection()
} return this
}
getLanguages(): Collection {
return this.feed ? this.feed.getLanguages() : new Collection() updateFilepath(): this {
} const channel = this.getChannel()
if (!channel) return this
hasLanguages() {
return this.feed ? this.feed.hasLanguages() : false this.filepath = `${channel.country.toLowerCase()}.m3u`
}
return this
hasLanguage(language: Language) { }
return this.feed ? this.feed.hasLanguage(language) : false
} updateTitle(): this {
const channel = this.getChannel()
getBroadcastAreaCodes(): Collection {
return this.feed ? this.feed.broadcastAreaCodes : new Collection() if (!channel) return this
}
const feed = this.getFeed()
isBroadcastInCity(city: City): boolean {
return this.feed ? this.feed.isBroadcastInCity(city) : false this.title = channel.name
} if (feed && !feed.is_main) {
this.title += ` ${feed.name}`
isBroadcastInSubdivision(subdivision: Subdivision): boolean { }
return this.feed ? this.feed.isBroadcastInSubdivision(subdivision) : false
} return this
}
isBroadcastInCountry(country: Country): boolean {
return this.feed ? this.feed.isBroadcastInCountry(country) : false normalizeURL() {
} this.url = normalizeURL(this.url)
}
isBroadcastInRegion(region: Region): boolean {
return this.feed ? this.feed.isBroadcastInRegion(region) : false getLogos(): Collection<sdk.Models.Logo> {
} const logos = super.getLogos()
isInternational(): boolean { if (logos.isEmpty()) return new Collection()
return this.feed ? this.feed.isInternational() : false
} function format(logo: sdk.Models.Logo): number {
const levelByFormat = { SVG: 0, PNG: 3, APNG: 1, WebP: 1, AVIF: 1, JPEG: 2, GIF: 1 }
getLogos(): Collection {
function format(logo: Logo): number { return logo.format ? levelByFormat[logo.format] : 0
const levelByFormat = { SVG: 0, PNG: 3, APNG: 1, WebP: 1, AVIF: 1, JPEG: 2, GIF: 1 } }
return logo.format ? levelByFormat[logo.format] : 0 function size(logo: sdk.Models.Logo): number {
} return Math.abs(512 - logo.width) + Math.abs(512 - logo.height)
}
function size(logo: Logo): number {
return Math.abs(512 - logo.width) + Math.abs(512 - logo.height) return logos.sortBy([format, size], ['desc', 'asc'], false)
} }
return this.logos.orderBy([format, size], ['desc', 'asc'], false) getFilepath(): string {
} return this.filepath || ''
}
getLogo(): Logo | undefined {
return this.getLogos().first() getFilename(): string {
} return path.basename(this.getFilepath())
}
hasLogo(): boolean {
return this.getLogos().notEmpty() getLine(): number {
} return this.line || -1
}
getLogoUrl(): string {
let logo: Logo | undefined getTvgId(): string {
if (this.tvgId) return this.tvgId
if (this.hasLogo()) logo = this.getLogo()
else logo = this?.channel?.getLogo() return this.getId()
}
return logo ? logo.url : ''
} getTvgLogo(): string {
const logo = this.getLogos().first()
getTitle(): string {
return this.title || '' return logo ? logo.url : ''
} }
getFullTitle(): string { getFullTitle(): string {
let title = `${this.getTitle()}` let title = `${this.title}`
if (this.getQuality()) { if (this.quality) {
title += ` (${this.getQuality()})` title += ` (${this.quality})`
} }
if (this.label) { if (this.label) {
title += ` [${this.label}]` title += ` [${this.label}]`
} }
return title return title
} }
getLabel(): string { toString(options: { public?: boolean } = {}) {
return this.label || '' options = { ...{ public: false }, ...options }
}
let output = `#EXTINF:-1 tvg-id="${this.getTvgId()}"`
getId(): string {
return this.id || '' if (options.public) {
} output += ` tvg-logo="${this.getTvgLogo()}" group-title="${this.groupTitle}"`
}
toJSON() {
return { if (this.referrer) {
channel: this.channelId || null, output += ` http-referrer="${this.referrer}"`
feed: this.feedId || null, }
title: this.title,
url: this.url, if (this.user_agent) {
referrer: this.referrer || null, output += ` http-user-agent="${this.user_agent}"`
user_agent: this.userAgent || null, }
quality: this.getQuality() || null
} output += `,${this.getFullTitle()}`
}
this.directives.forEach((prop: string) => {
toString(options: { public: boolean }) { output += `\r\n${prop}`
let output = `#EXTINF:-1 tvg-id="${this.getId()}"` })
if (options.public) { output += `\r\n${this.url}`
output += ` tvg-logo="${this.getLogoUrl()}" group-title="${this.groupTitle}"`
} return output
}
if (this.referrer) {
output += ` http-referrer="${this.referrer}"` toObject(): sdk.Types.StreamData {
} let feedId = this.feed
if (!feedId) {
if (this.userAgent) { const feed = this.getFeed()
output += ` http-user-agent="${this.userAgent}"` if (feed) feedId = feed.id
} }
output += `,${this.getFullTitle()}` return {
channel: this.channel,
this.directives.forEach((prop: string) => { feed: feedId,
output += `\r\n${prop}` title: this.title,
}) url: this.url,
quality: this.quality,
output += `\r\n${this.url}` user_agent: this.user_agent,
referrer: this.referrer
return output }
} }
}
clone(): Stream {
function escapeRegExp(text) { return Object.assign(Object.create(Object.getPrototypeOf(this)), this)
return text.replace(/[-[\]{}()*+?.,\\^$|#\s]/g, '\\$&') }
} }
function parseQuality(quality: string | null): {
verticalResolution: number | null
isInterlaced: boolean | null
} {
if (!quality) return { verticalResolution: null, isInterlaced: null }
const [, verticalResolutionString] = quality.match(/^(\d+)/) || [null, undefined]
const isInterlaced = /i$/i.test(quality)
let verticalResolution = 0
if (verticalResolutionString) verticalResolution = parseInt(verticalResolutionString)
return { verticalResolution, isInterlaced }
}

83
scripts/models/subdivision.ts
View File

@@ -1,83 +0,0 @@
import { SubdivisionData, SubdivisionSerializedData } from '../types/subdivision'
import { Dictionary, Collection } from '@freearhey/core'
import { Country, Region } from '.'
export class Subdivision {
code: string
name: string
countryCode: string
country?: Country
parentCode?: string
parent?: Subdivision
regions?: Collection
cities?: Collection
constructor(data?: SubdivisionData) {
if (!data) return
this.code = data.code
this.name = data.name
this.countryCode = data.country
this.parentCode = data.parent || undefined
}
withCountry(countriesKeyByCode: Dictionary): this {
this.country = countriesKeyByCode.get(this.countryCode)
return this
}
withRegions(regions: Collection): this {
this.regions = regions.filter((region: Region) =>
region.countryCodes.includes(this.countryCode)
)
return this
}
withCities(citiesGroupedBySubdivisionCode: Dictionary): this {
this.cities = new Collection(citiesGroupedBySubdivisionCode.get(this.code))
return this
}
withParent(subdivisionsKeyByCode: Dictionary): this {
if (!this.parentCode) return this
this.parent = subdivisionsKeyByCode.get(this.parentCode)
return this
}
getRegions(): Collection {
if (!this.regions) return new Collection()
return this.regions
}
getCities(): Collection {
if (!this.cities) return new Collection()
return this.cities
}
serialize(): SubdivisionSerializedData {
return {
code: this.code,
name: this.name,
countryCode: this.countryCode,
country: this.country ? this.country.serialize() : undefined,
parentCode: this.parentCode || null
}
}
deserialize(data: SubdivisionSerializedData): this {
this.code = data.code
this.name = data.name
this.countryCode = data.countryCode
this.country = data.country ? new Country().deserialize(data.country) : undefined
this.parentCode = data.parentCode || undefined
return this
}
}

30
scripts/models/timezone.ts
View File

@@ -1,30 +0,0 @@
import { Collection, Dictionary } from '@freearhey/core'
type TimezoneData = {
id: string
utc_offset: string
countries: string[]
}
export class Timezone {
id: string
utcOffset: string
countryCodes: Collection
countries?: Collection
constructor(data: TimezoneData) {
this.id = data.id
this.utcOffset = data.utc_offset
this.countryCodes = new Collection(data.countries)
}
withCountries(countriesKeyByCode: Dictionary): this {
this.countries = this.countryCodes.map((code: string) => countriesKeyByCode.get(code))
return this
}
getCountries(): Collection {
return this.countries || new Collection()
}
}

119
scripts/tables/categoriesTable.ts
View File

@@ -1,56 +1,63 @@
import { Storage, Collection, File, Dictionary } from '@freearhey/core' import { HTMLTable, HTMLTableItem, LogParser, LogItem, HTMLTableColumn } from '../core'
import { HTMLTable, LogParser, LogItem } from '../core' import { Storage, File } from '@freearhey/storage-js'
import { LOGS_DIR, README_DIR } from '../constants' import { LOGS_DIR, README_DIR } from '../constants'
import { Category } from '../models' import { Collection } from '@freearhey/core'
import { Table } from './table' import * as sdk from '@iptv-org/sdk'
import { Table } from './table'
type CategoriesTableProps = { import { data } from '../api'
categoriesKeyById: Dictionary
} export class CategoriesTable implements Table {
async create() {
export class CategoriesTable implements Table { const parser = new LogParser()
categoriesKeyById: Dictionary const logsStorage = new Storage(LOGS_DIR)
const generatorsLog = await logsStorage.load('generators.log')
constructor({ categoriesKeyById }: CategoriesTableProps) {
this.categoriesKeyById = categoriesKeyById let items = new Collection<HTMLTableItem>()
} parser
.parse(generatorsLog)
async make() { .filter((logItem: LogItem) => logItem.type === 'category')
const parser = new LogParser() .forEach((logItem: LogItem) => {
const logsStorage = new Storage(LOGS_DIR) if (logItem.filepath.includes('undefined')) {
const generatorsLog = await logsStorage.load('generators.log') items.add([
'ZZ',
let items = new Collection() 'Undefined',
parser logItem.count.toString(),
.parse(generatorsLog) `<code>https://iptv-org.github.io/iptv/${logItem.filepath}</code>`
.filter((logItem: LogItem) => logItem.type === 'category') ])
.forEach((logItem: LogItem) => {
const file = new File(logItem.filepath) return
const categoryId = file.name() }
const category: Category = this.categoriesKeyById.get(categoryId)
const file = new File(logItem.filepath)
items.add([ const categoryId = file.name()
category ? category.name : 'ZZ', const category: sdk.Models.Category | undefined = data.categoriesKeyById.get(categoryId)
category ? category.name : 'Undefined',
logItem.count, if (!category) return
`<code>https://iptv-org.github.io/iptv/${logItem.filepath}</code>`
]) items.add([
}) category.name,
category.name,
items = items logItem.count.toString(),
.orderBy(item => item[0]) `<code>https://iptv-org.github.io/iptv/${logItem.filepath}</code>`
.map(item => { ])
item.shift() })
return item
}) items = items
.sortBy(item => item[0])
const table = new HTMLTable(items.all(), [ .map(item => {
{ name: 'Category' }, item.shift()
{ name: 'Channels', align: 'right' }, return item
{ name: 'Playlist', nowrap: true } })
])
const columns = new Collection<HTMLTableColumn>([
const readmeStorage = new Storage(README_DIR) { name: 'Category' },
await readmeStorage.save('_categories.md', table.toString()) { name: 'Channels', align: 'right' },
} { name: 'Playlist', nowrap: true }
} ])
const table = new HTMLTable(items, columns)
const readmeStorage = new Storage(README_DIR)
await readmeStorage.save('_categories.md', table.toString())
}
}

365
scripts/tables/countriesTable.ts
View File

@@ -1,189 +1,176 @@
import { Storage, Collection, Dictionary } from '@freearhey/core' import { LOGS_DIR, README_DIR } from '../constants'
import { City, Country, Subdivision } from '../models' import { Storage } from '@freearhey/storage-js'
import { LOGS_DIR, README_DIR } from '../constants' import { Collection } from '@freearhey/core'
import { LogParser, LogItem } from '../core' import { LogParser, LogItem } from '../core'
import { Table } from './table' import * as sdk from '@iptv-org/sdk'
import { Table } from './table'
type CountriesTableProps = { import { data } from '../api'
countriesKeyByCode: Dictionary
subdivisionsKeyByCode: Dictionary type ListItem = {
countries: Collection index: string
subdivisions: Collection count: number
cities: Collection link: string
} name: string
children: Collection<ListItem>
export class CountriesTable implements Table { }
countriesKeyByCode: Dictionary
subdivisionsKeyByCode: Dictionary export class CountriesTable implements Table {
countries: Collection async create() {
subdivisions: Collection const parser = new LogParser()
cities: Collection const logsStorage = new Storage(LOGS_DIR)
const generatorsLog = await logsStorage.load('generators.log')
constructor({ const parsed = parser.parse(generatorsLog)
countriesKeyByCode, const logCountries = parsed.filter((logItem: LogItem) => logItem.type === 'country')
subdivisionsKeyByCode, const logSubdivisions = parsed.filter((logItem: LogItem) => logItem.type === 'subdivision')
countries, const logCities = parsed.filter((logItem: LogItem) => logItem.type === 'city')
subdivisions,
cities let items = new Collection()
}: CountriesTableProps) { data.countries.forEach((country: sdk.Models.Country) => {
this.countriesKeyByCode = countriesKeyByCode const countryCode = country.code
this.subdivisionsKeyByCode = subdivisionsKeyByCode const countriesLogItem = logCountries.find(
this.countries = countries (logItem: LogItem) => logItem.filepath === `countries/${countryCode.toLowerCase()}.m3u`
this.subdivisions = subdivisions )
this.cities = cities
} const countryItem: ListItem = {
index: country.name,
async make() { count: 0,
const parser = new LogParser() link: `https://iptv-org.github.io/iptv/countries/${countryCode.toLowerCase()}.m3u`,
const logsStorage = new Storage(LOGS_DIR) name: `${country.flag} ${country.name}`,
const generatorsLog = await logsStorage.load('generators.log') children: new Collection()
const parsed = parser.parse(generatorsLog) }
const logCountries = parsed.filter((logItem: LogItem) => logItem.type === 'country')
const logSubdivisions = parsed.filter((logItem: LogItem) => logItem.type === 'subdivision') if (countriesLogItem) {
const logCities = parsed.filter((logItem: LogItem) => logItem.type === 'city') countryItem.count = countriesLogItem.count
}
let items = new Collection()
this.countries.forEach((country: Country) => { const countrySubdivisions = data.subdivisions.filter(
const countriesLogItem = logCountries.find( (subdivision: sdk.Models.Subdivision) => subdivision.country === countryCode
(logItem: LogItem) => logItem.filepath === `countries/${country.code.toLowerCase()}.m3u` )
) const countryCities = data.cities.filter(
(city: sdk.Models.City) => city.country === countryCode
const countryItem = { )
index: country.name, if (countrySubdivisions.isNotEmpty()) {
count: 0, data.subdivisions.forEach((subdivision: sdk.Models.Subdivision) => {
link: `https://iptv-org.github.io/iptv/countries/${country.code.toLowerCase()}.m3u`, if (subdivision.country !== countryCode) return
name: `${country.flag} ${country.name}`,
children: new Collection() const subdivisionCode = subdivision.code
} const subdivisionCities = countryCities.filter(
(city: sdk.Models.City) =>
if (countriesLogItem) { (city.subdivision && city.subdivision === subdivisionCode) ||
countryItem.count = countriesLogItem.count city.country === subdivision.country
} )
const subdivisionsLogItem = logSubdivisions.find(
const countrySubdivisions = this.subdivisions.filter( (logItem: LogItem) =>
(subdivision: Subdivision) => subdivision.countryCode === country.code logItem.filepath === `subdivisions/${subdivisionCode.toLowerCase()}.m3u`
) )
const countryCities = this.cities.filter((city: City) => city.countryCode === country.code)
if (countrySubdivisions.notEmpty()) { const subdivisionItem: ListItem = {
this.subdivisions.forEach((subdivision: Subdivision) => { index: subdivision.name,
if (subdivision.countryCode !== country.code) return name: subdivision.name,
const subdivisionCities = countryCities.filter( count: 0,
(city: City) => link: `https://iptv-org.github.io/iptv/subdivisions/${subdivisionCode.toLowerCase()}.m3u`,
(city.subdivisionCode && city.subdivisionCode === subdivision.code) || children: new Collection<ListItem>()
city.countryCode === subdivision.countryCode }
)
const subdivisionsLogItem = logSubdivisions.find( if (subdivisionsLogItem) {
(logItem: LogItem) => subdivisionItem.count = subdivisionsLogItem.count
logItem.filepath === `subdivisions/${subdivision.code.toLowerCase()}.m3u` }
)
subdivisionCities.forEach((city: sdk.Models.City) => {
const subdivisionItem = { if (city.country !== countryCode || city.subdivision !== subdivisionCode) return
index: subdivision.name, const citiesLogItem = logCities.find(
name: subdivision.name, (logItem: LogItem) => logItem.filepath === `cities/${city.code.toLowerCase()}.m3u`
count: 0, )
link: `https://iptv-org.github.io/iptv/subdivisions/${subdivision.code.toLowerCase()}.m3u`,
children: new Collection() if (!citiesLogItem) return
}
subdivisionItem.children.add({
if (subdivisionsLogItem) { index: city.name,
subdivisionItem.count = subdivisionsLogItem.count name: city.name,
} count: citiesLogItem.count,
link: `https://iptv-org.github.io/iptv/${citiesLogItem.filepath}`,
subdivisionCities.forEach((city: City) => { children: new Collection<ListItem>()
if (city.countryCode !== country.code || city.subdivisionCode !== subdivision.code) })
return })
const citiesLogItem = logCities.find(
(logItem: LogItem) => logItem.filepath === `cities/${city.code.toLowerCase()}.m3u` if (subdivisionItem.count > 0 || subdivisionItem.children.isNotEmpty()) {
) countryItem.children.add(subdivisionItem)
}
if (!citiesLogItem) return })
} else if (countryCities.isNotEmpty()) {
subdivisionItem.children.add({ countryCities.forEach((city: sdk.Models.City) => {
index: city.name, const citiesLogItem = logCities.find(
name: city.name, (logItem: LogItem) => logItem.filepath === `cities/${city.code.toLowerCase()}.m3u`
count: citiesLogItem.count, )
link: `https://iptv-org.github.io/iptv/${citiesLogItem.filepath}`
}) if (!citiesLogItem) return
})
countryItem.children.add({
if (subdivisionItem.count > 0 || subdivisionItem.children.notEmpty()) { index: city.name,
countryItem.children.add(subdivisionItem) name: city.name,
} count: citiesLogItem.count,
}) link: `https://iptv-org.github.io/iptv/${citiesLogItem.filepath}`,
} else if (countryCities.notEmpty()) { children: new Collection()
countryCities.forEach((city: City) => { })
const citiesLogItem = logCities.find( })
(logItem: LogItem) => logItem.filepath === `cities/${city.code.toLowerCase()}.m3u` }
)
if (countryItem.count > 0 || countryItem.children.isNotEmpty()) {
if (!citiesLogItem) return items.add(countryItem)
}
countryItem.children.add({ })
index: city.name,
name: city.name, const internationalLogItem = logCountries.find(
count: citiesLogItem.count, (logItem: LogItem) => logItem.filepath === 'countries/int.m3u'
link: `https://iptv-org.github.io/iptv/${citiesLogItem.filepath}`, )
children: new Collection()
}) if (internationalLogItem) {
}) items.add({
} index: 'ZZ',
name: '🌐 International',
if (countryItem.count > 0 || countryItem.children.notEmpty()) { count: internationalLogItem.count,
items.add(countryItem) link: `https://iptv-org.github.io/iptv/${internationalLogItem.filepath}`,
} children: new Collection()
}) })
}
const internationalLogItem = logCountries.find(
(logItem: LogItem) => logItem.filepath === 'countries/int.m3u' const undefinedLogItem = logCountries.find(
) (logItem: LogItem) => logItem.filepath === 'countries/undefined.m3u'
)
if (internationalLogItem) {
items.push({ if (undefinedLogItem) {
index: 'ZZ', items.add({
name: '🌐 International', index: 'ZZZ',
count: internationalLogItem.count, name: 'Undefined',
link: `https://iptv-org.github.io/iptv/${internationalLogItem.filepath}`, count: undefinedLogItem.count,
children: new Collection() link: `https://iptv-org.github.io/iptv/${undefinedLogItem.filepath}`,
}) children: new Collection()
} })
}
const undefinedLogItem = logCountries.find(
(logItem: LogItem) => logItem.filepath === 'countries/undefined.m3u' items = items.sortBy(item => item.index)
)
const output = items
if (undefinedLogItem) { .map((item: ListItem) => {
items.push({ let row = `- ${item.name} <code>${item.link}</code>`
index: 'ZZZ',
name: 'Undefined', item.children
count: undefinedLogItem.count, .sortBy((item: ListItem) => item.index)
link: `https://iptv-org.github.io/iptv/${undefinedLogItem.filepath}`, .forEach((item: ListItem) => {
children: new Collection() row += `\r\n - ${item.name} <code>${item.link}</code>`
})
} item.children
.sortBy((item: ListItem) => item.index)
items = items.orderBy(item => item.index) .forEach((item: ListItem) => {
row += `\r\n - ${item.name} <code>${item.link}</code>`
const output = items })
.map(item => { })
let row = `- ${item.name} <code>${item.link}</code>`
return row
item.children })
.orderBy(item => item.index) .join('\r\n')
.forEach(item => {
row += `\r\n - ${item.name} <code>${item.link}</code>` const readmeStorage = new Storage(README_DIR)
await readmeStorage.save('_countries.md', output)
item.children }
.orderBy(item => item.index) }
.forEach(item => {
row += `\r\n - ${item.name} <code>${item.link}</code>`
})
})
return row
})
.join('\r\n')
const readmeStorage = new Storage(README_DIR)
await readmeStorage.save('_countries.md', output)
}
}

8
scripts/tables/index.ts
View File

@@ -1,4 +1,4 @@
export * from './categoriesTable' export * from './categoriesTable'
export * from './countriesTable' export * from './countriesTable'
export * from './languagesTable' export * from './languagesTable'
export * from './regionsTable' export * from './regionsTable'

119
scripts/tables/languagesTable.ts
View File

@@ -1,56 +1,63 @@
import { Storage, Collection, File, Dictionary } from '@freearhey/core' import { HTMLTable, LogParser, LogItem, HTMLTableColumn, HTMLTableItem } from '../core'
import { HTMLTable, LogParser, LogItem } from '../core' import { Storage, File } from '@freearhey/storage-js'
import { LOGS_DIR, README_DIR } from '../constants' import { LOGS_DIR, README_DIR } from '../constants'
import { Language } from '../models' import { Collection } from '@freearhey/core'
import { Table } from './table' import * as sdk from '@iptv-org/sdk'
import { Table } from './table'
type LanguagesTableProps = { import { data } from '../api'
languagesKeyByCode: Dictionary
} export class LanguagesTable implements Table {
async create() {
export class LanguagesTable implements Table { const parser = new LogParser()
languagesKeyByCode: Dictionary const logsStorage = new Storage(LOGS_DIR)
const generatorsLog = await logsStorage.load('generators.log')
constructor({ languagesKeyByCode }: LanguagesTableProps) {
this.languagesKeyByCode = languagesKeyByCode let items = new Collection<HTMLTableItem>()
} parser
.parse(generatorsLog)
async make() { .filter((logItem: LogItem) => logItem.type === 'language')
const parser = new LogParser() .forEach((logItem: LogItem) => {
const logsStorage = new Storage(LOGS_DIR) if (logItem.filepath.includes('undefined')) {
const generatorsLog = await logsStorage.load('generators.log') items.add([
'ZZ',
let data = new Collection() 'Undefined',
parser logItem.count.toString(),
.parse(generatorsLog) `<code>https://iptv-org.github.io/iptv/${logItem.filepath}</code>`
.filter((logItem: LogItem) => logItem.type === 'language') ])
.forEach((logItem: LogItem) => {
const file = new File(logItem.filepath) return
const languageCode = file.name() }
const language: Language = this.languagesKeyByCode.get(languageCode)
const file = new File(logItem.filepath)
data.add([ const languageCode = file.name()
language ? language.name : 'ZZ', const language: sdk.Models.Language | undefined = data.languagesKeyByCode.get(languageCode)
language ? language.name : 'Undefined',
logItem.count, if (!language) return
`<code>https://iptv-org.github.io/iptv/${logItem.filepath}</code>`
]) items.add([
}) language.name,
language.name,
data = data logItem.count.toString(),
.orderBy(item => item[0]) `<code>https://iptv-org.github.io/iptv/${logItem.filepath}</code>`
.map(item => { ])
item.shift() })
return item
}) items = items
.sortBy(item => item[0])
const table = new HTMLTable(data.all(), [ .map(item => {
{ name: 'Language', align: 'left' }, item.shift()
{ name: 'Channels', align: 'right' }, return item
{ name: 'Playlist', align: 'left', nowrap: true } })
])
const columns = new Collection<HTMLTableColumn>([
const readmeStorage = new Storage(README_DIR) { name: 'Language', align: 'left' },
await readmeStorage.save('_languages.md', table.toString()) { name: 'Channels', align: 'right' },
} { name: 'Playlist', align: 'left', nowrap: true }
} ])
const table = new HTMLTable(items, columns)
const readmeStorage = new Storage(README_DIR)
await readmeStorage.save('_languages.md', table.toString())
}
}

101
scripts/tables/regionsTable.ts
View File

@@ -1,52 +1,49 @@
import { Storage, Collection } from '@freearhey/core' import { LOGS_DIR, README_DIR } from '../constants'
import { LogParser, LogItem } from '../core' import { Storage } from '@freearhey/storage-js'
import { LOGS_DIR, README_DIR } from '../constants' import { LogParser, LogItem } from '../core'
import { Region } from '../models' import { Collection } from '@freearhey/core'
import { Table } from './table' import * as sdk from '@iptv-org/sdk'
import { Table } from './table'
type RegionsTableProps = { import { data } from '../api'
regions: Collection
} type ListItem = {
name: string
export class RegionsTable implements Table { count: number
regions: Collection link: string
}
constructor({ regions }: RegionsTableProps) {
this.regions = regions export class RegionsTable implements Table {
} async create() {
const parser = new LogParser()
async make() { const logsStorage = new Storage(LOGS_DIR)
const parser = new LogParser() const generatorsLog = await logsStorage.load('generators.log')
const logsStorage = new Storage(LOGS_DIR) const parsed = parser.parse(generatorsLog)
const generatorsLog = await logsStorage.load('generators.log') const logRegions = parsed.filter((logItem: LogItem) => logItem.type === 'region')
const parsed = parser.parse(generatorsLog)
const logRegions = parsed.filter((logItem: LogItem) => logItem.type === 'region') let items = new Collection<ListItem>()
data.regions.forEach((region: sdk.Models.Region) => {
let items = new Collection() const logItem = logRegions.find(
this.regions.forEach((region: Region) => { (logItem: LogItem) => logItem.filepath === `regions/${region.code.toLowerCase()}.m3u`
const logItem = logRegions.find( )
(logItem: LogItem) => logItem.filepath === `regions/${region.code.toLowerCase()}.m3u`
) if (!logItem) return
if (!logItem) return items.add({
name: region.name,
items.add({ count: logItem.count,
index: region.name, link: `https://iptv-org.github.io/iptv/${logItem.filepath}`
name: region.name, })
count: logItem.count, })
link: `https://iptv-org.github.io/iptv/${logItem.filepath}`
}) items = items.sortBy(item => item.name)
})
const output = items
items = items.orderBy(item => item.index) .map(item => {
return `- ${item.name} <code>${item.link}</code>`
const output = items })
.map(item => { .join('\r\n')
return `- ${item.name} <code>${item.link}</code>`
}) const readmeStorage = new Storage(README_DIR)
.join('\r\n') await readmeStorage.save('_regions.md', output)
}
const readmeStorage = new Storage(README_DIR) }
await readmeStorage.save('_regions.md', output)
}
}

6
scripts/tables/table.ts
View File

@@ -1,3 +1,3 @@
export interface Table { export interface Table {
make(): void create(): void
} }

5
scripts/types/blocklistRecord.d.ts vendored
View File

@@ -1,5 +0,0 @@
export type BlocklistRecordData = {
channel: string
reason: string
ref: string
}

9
scripts/types/category.d.ts vendored
View File

@@ -1,9 +0,0 @@
export type CategorySerializedData = {
id: string
name: string
}
export type CategoryData = {
id: string
name: string
}

50
scripts/types/channel.d.ts vendored
View File

@@ -1,50 +0,0 @@
import { Collection } from '@freearhey/core'
import type { CountrySerializedData } from './country'
import type { SubdivisionSerializedData } from './subdivision'
import type { CategorySerializedData } from './category'
export type ChannelSerializedData = {
id: string
name: string
altNames: string[]
network?: string
owners: string[]
countryCode: string
country?: CountrySerializedData
subdivisionCode?: string
subdivision?: SubdivisionSerializedData
cityName?: string
categoryIds: string[]
categories?: CategorySerializedData[]
isNSFW: boolean
launched?: string
closed?: string
replacedBy?: string
website?: string
}
export type ChannelData = {
id: string
name: string
alt_names: string[]
network: string
owners: Collection
country: string
subdivision: string
city: string
categories: Collection
is_nsfw: boolean
launched: string
closed: string
replaced_by: string
website: string
}
export type ChannelSearchableData = {
id: string
name: string
altNames: string[]
guideNames: string[]
streamTitles: string[]
feedFullNames: string[]
}

20
scripts/types/city.d.ts vendored
View File

@@ -1,20 +0,0 @@
import { CountrySerializedData } from './country'
import { SubdivisionSerializedData } from './subdivision'
export type CitySerializedData = {
code: string
name: string
countryCode: string
country?: CountrySerializedData
subdivisionCode: string | null
subdivision?: SubdivisionSerializedData
wikidataId: string
}
export type CityData = {
code: string
name: string
country: string
subdivision: string | null
wikidata_id: string
}

20
scripts/types/country.d.ts vendored
View File

@@ -1,20 +0,0 @@
import type { LanguageSerializedData } from './language'
import type { SubdivisionSerializedData } from './subdivision'
import type { RegionSerializedData } from './region'
export type CountrySerializedData = {
code: string
name: string
flag: string
languageCode: string
language: LanguageSerializedData | null
subdivisions: SubdivisionSerializedData[]
regions: RegionSerializedData[]
}
export type CountryData = {
code: string
name: string
lang: string
flag: string
}

21
scripts/types/dataLoader.d.ts vendored
View File

@@ -1,21 +0,0 @@
import { Storage } from '@freearhey/core'
export type DataLoaderProps = {
storage: Storage
}
export type DataLoaderData = {
countries: object | object[]
regions: object | object[]
subdivisions: object | object[]
languages: object | object[]
categories: object | object[]
blocklist: object | object[]
channels: object | object[]
feeds: object | object[]
logos: object | object[]
timezones: object | object[]
guides: object | object[]
streams: object | object[]
cities: object | object[]
}

31
scripts/types/dataProcessor.d.ts vendored
View File

@@ -1,31 +0,0 @@
import { Collection, Dictionary } from '@freearhey/core'
export type DataProcessorData = {
blocklistRecordsGroupedByChannelId: Dictionary
subdivisionsGroupedByCountryCode: Dictionary
feedsGroupedByChannelId: Dictionary
guidesGroupedByStreamId: Dictionary
logosGroupedByStreamId: Dictionary
subdivisionsKeyByCode: Dictionary
countriesKeyByCode: Dictionary
languagesKeyByCode: Dictionary
streamsGroupedById: Dictionary
categoriesKeyById: Dictionary
timezonesKeyById: Dictionary
regionsKeyByCode: Dictionary
blocklistRecords: Collection
channelsKeyById: Dictionary
citiesKeyByCode: Dictionary
subdivisions: Collection
categories: Collection
countries: Collection
languages: Collection
timezones: Collection
channels: Collection
regions: Collection
streams: Collection
cities: Collection
guides: Collection
feeds: Collection
logos: Collection
}

10
scripts/types/feed.d.ts vendored
View File

@@ -1,10 +0,0 @@
export type FeedData = {
channel: string
id: string
name: string
is_main: boolean
broadcast_area: string[]
languages: string[]
timezones: string[]
video_format: string
}

17
scripts/types/guide.d.ts vendored
View File

@@ -1,17 +0,0 @@
export type GuideSerializedData = {
channelId?: string
feedId?: string
siteDomain: string
siteId: string
siteName: string
languageCode: string
}
export type GuideData = {
channel: string
feed: string
site: string
site_id: string
site_name: string
lang: string
}

9
scripts/types/language.d.ts vendored
View File

@@ -1,9 +0,0 @@
export type LanguageSerializedData = {
code: string
name: string
}
export type LanguageData = {
code: string
name: string
}

Some files were not shown because too many files have changed in this diff Show More