Feature #747

New backends management

Added by Romain Bignon over 4 years ago. Updated over 4 years ago.

Status:Resolved Start:2012-01-01
Priority:Normal Due date:
Assigned to:Romain Bignon % Done:

100%

Category:Core / Backends management Spent time: -
Target version:0.a
Module: Branch:

Description

Instead of distributing backends with Weboob, we should provide a way to have weboob repositories where backends are and where weboob applications will find and download new backends. It will also let the ability to support upgrade of backends.

A weboob repository should be represented by a directory containing a file with all available backends (for example http://updates.weboob.org/%(version)s/main/modules.list) with the following syntax:

[DEFAULT]
last_update = 123132546
name = main
maintainer = Weboob team

[dlfp]
version = 201201031257
capabilities = ICapMessages ICapMessagesPost ICapContent ICap
description = Da Linux French Page
maintainer = Romain Bignon
email = romain@weboob.org
license = AGPLv3+
icon = http://linuxfr.org/favicon.ico
urls = https?.//(www\.)?linuxfr\.org/.*

The repository's tree should be something like:

/0.a/
  modules.list
  dlfp.tar.gz
  aum.tar.gz
  ...

Client-side, on the first run of Weboob, it creates a file named ~/.weboob/sources.list containing list of repositories:

http://updates.weboob.org/%(version)s/main/
# To enable NSFW backends, uncomment the following line:
#http://updates.weboob.org/%(version)s/nsfw/

Then, it downloads all modules.list files in ~/.weboob/repositories/.

weboob-config will provide the following commands:

  • modules: list all available modules in repositories.
  • add: create a new backend. if the module isn't installed, it downloads it.
  • remove: remove a backend.
  • update: download new versions of repositories modules.list files, and upgrade modules with new version available.
  • list: list all backends

Note: I think we should use explicitly module to designate the python module to weboob, and backend only an instance of a module.

Downloaded modules are put into ~/.weboob/modules/. When there is a bug in a backend, the application suggests to see if there is any available new version. It should also be interesting to provide a way to automatically send a bug report.


Related issues

precedes weboob - Feature #755: Repositories: sign packages with GPG Resolved 2012-01-18 2012-01-30
precedes weboob - Feature #756: Download icons asynchronously Resolved 2012-01-18
precedes weboob - Feature #757: Associate URLs regexps to modules Assigned 2012-01-18

Associated revisions

Revision 14a7a1d36296190ae771cd62ce84d335b1dd0ac6
Added by Romain Bignon over 4 years ago

support repositories to manage backends (closes #747)

History

Updated by Romain Bignon over 4 years ago

I would like to add a parameter url to modules in modules.list to give a regexp of supported URLs.

For example, with youtube:

urls = https?://(youtube\.com|youtu\.be).*

When doing something like:

$ videoob play http://youtube.com/watch?v=5ILXM4u-N18

It will:

  1. Load only backends which support this kind of url
  2. If the isn't any backend configured, ask user to create it

Updated by Romain Bignon over 4 years ago

Here is the guideline when developing on backends with this new feature.

It will be needed to add in ~/.weboob/sources.list a line to set the local path of your module(s):

file:///home/rom1/src/weboob/modules/

Then, you can edit module's sources, and to apply changes to modules into ~/.weboob/, you'll need to do:

weboob-config update --local

It regenerates /home/rom1/src/weboob/modules/modules.list and as your module has been modified, it copies it into ~/.weboob/modules/.

I don't know if it is annoying to impose usage of a command to update your module.

Another method would be to consider that if there is a local path in sources.list, we do not download modules from this repository in ~/.weboob/modules/ but we load them in /home/rom1/src/weboob/modules/.

Any idea?

Updated by Laurent Bachelier over 4 years ago

I think having to type a command every time a backend is modified is going to be annoying, and we will always forget if we did run it or not.

So I would prefer the second method.

Updated by Laurent Bachelier over 4 years ago

To check for the authenticity of the backends, we could use GPG.

I propose:
http://packages.debian.org/squeeze/python-gpgme http://pypi.python.org/pypi/pygpgme

Advantages over py-gnupg: has a commit less than 10 years ago. http://pypi.python.org/pypi/GnuPGInterface/0.3.2
Advantages over python-gnupg: is packages in Debian. http://pypi.python.org/pypi/python-gnupg/0.2.8

I'm not convinced we need a lib though, maybe calling gpg --verify is easy enough. For Windows, we can distribute a binary of gpg (according to http://www.red-dove.com/python_gnupg/index.html you only need two files). Or we could make the check able to be disabled, for people who like to live dangerously (i.e. Windows users).

Updated by Romain Bignon over 4 years ago

  • Status changed from Assigned to In progress
  • % Done changed from 0 to 50

Updated by Romain Bignon over 4 years ago

  • Status changed from In progress to Resolved
  • % Done changed from 50 to 100

Also available in: Atom PDF