Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix & feat: use sqlite to track upload queue and add re-upload when last upload task failed #234

Merged
merged 5 commits into from
Mar 31, 2025

Conversation

LimLLL
Copy link
Contributor

@LimLLL LimLLL commented Mar 25, 2025

Description

  1. Use SQLite to track the upload queue instead of writing info to a file.
  2. Create unique upload config.yaml for each video clip since re-upload needs the info about which video failed and its corresponding config file.
  3. add a config var IGNORE_ROOM_TITLE, allowing the user to control whether to include room title in the video title which can partial avoid create a new video when streamer changed the streaming title.

Related Issues

#224
#225

Who Can Review?

@timerring

Checklist

  • Code has been reviewed
  • Code complies with the project's code standards and best practices
  • Code has passed all tests
  • Code does not affect the normal use of existing features
  • Code has been commented properly
  • Documentation has been updated (if applicable)
  • Demo/checkpoint has been attached (if applicable)

@LimLLL
Copy link
Contributor Author

LimLLL commented Mar 25, 2025

解决分p这个问题比较暴力目前,感觉可以就是判断如果不包含直播标题在视频标题里的话,那么就放到分p的视频文件名里,这样会直观一些?

@timerring
Copy link
Owner

稍等,目前电脑不在跟前,明天讨论一下具体的解决方案🤔

@LimLLL
Copy link
Contributor Author

LimLLL commented Mar 27, 2025

稍等,目前电脑不在跟前,明天讨论一下具体的解决方案🤔

抱歉,这两天没咋上github,需要的话可以直接找我tg哈,https://t.me/si_shu_ba_wang

@timerring
Copy link
Owner

好的,非常感谢,我今明两天会接入两个我最近制作的库,之后就能捋一下流程了。

Copy link
Owner

@timerring timerring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, wait for test.

Comment on lines 40 to 52
data_tmp = datetime.strptime(date, '%Y%m%d')
year = data_tmp.strftime('%Y')
month = data_tmp.strftime('%m')
day = data_tmp.strftime('%d')

human_readable_date = f"{year}年{month}月{day}日"
if GPU_EXIST:
new_title = "【弹幕+字幕】" + artist + "直播回放-" + date + "-" + title
new_title = "【弹幕+字幕】" + artist + "直播回放-" + human_readable_date + "-" + title
else:
new_title = "【弹幕】" + artist + "直播回放-" + date + "-" + title
new_title = "【弹幕】" + artist + "直播回放-" + human_readable_date + "-" + title

if IGNORE_ROOM_TITLE:
new_title = new_title.replace('-'+title, "")
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be we can shelve this config, I have exported this parameter.

Copy link
Owner

@timerring timerring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@timerring timerring self-assigned this Mar 31, 2025
@timerring timerring merged commit 4e7be3c into timerring:main Mar 31, 2025
@timerring
Copy link
Owner

老哥,我目前下个版本打算用 webhook 的方式监听端口,因为我发现 brlec 有写 webhook 的反馈。稍后我测试后 push 一个简单的模板。

另外 yaml 的方式可能还会修改,因为我 bilitool 的所有功能已经重构出来了,以后整个上传方式应该可以更加灵活。

@LimLLL
Copy link
Contributor Author

LimLLL commented Mar 31, 2025

老哥,我目前下个版本打算用 webhook 的方式监听端口,因为我发现 brlec 有写 webhook 的反馈。稍后我测试后 push 一个简单的模板。

另外 yaml 的方式可能还会修改,因为我 bilitool 的所有功能已经重构出来了,以后整个上传方式应该可以更加灵活。

收到,之后你这边融合完上下游工具,我看再写个前端界面,方便用户查看一些数据?例如哪些正在scan(把scan数据也写到sqlite里),哪些upload失败了正在重试,哪些已经彻底失败然后用户可以复制指令手动上传试试。不知道老哥会不会觉得多此一举😂

@timerring
Copy link
Owner

可以的老哥,在没能自动修复并上传之前,让大家通过前端能直接了解哪些可以手动修复也挺有意义的。🤔

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants