News

Weboob 0.9

Added by Romain Bignon 4 months ago

After several months, Weboob 0.9 has been released.

This new version introduces mainly bugfixes, but also some key features, like the new CapBugTracker supported by Redmine with the boobtracker application, or the roadmap feature introduced into traveloob.

Also, a lot of minor backends have been added to Weboob. It beg the question of a new backends management, where it would be easier to create smaller backends, and to put them in an organized tree in weboob/backends.

There are now 54 backends and 24 applications.

Credits

ChangeLog

General

Backend: Arte

  • Set duration on video results.

Backend: AuM

  • Rewrite to use the mobile API.

Backend: BNPorc

  • Get operation categories.

Backend: CrAgr

  • Get full history.
  • Optimized operations retrieval for CA Centre.
  • Fix: parsing accounts list.

Backend: CreditMutuel

  • Maine-Anjou, Basse-Normandie bank added.

Backend: Dailymotion

  • Fix: parsing duration.
  • Fix: parsing of lives in search results.

Backend: DLFP

  • Fix: disconnection URL needs to be called with POST instead of GET.
  • Fix: do tests on alpha.linuxfr.org.
  • Fix: relevance on news/diaries.

Backend: Gazelle

  • Handle login errors.
  • Fix: parsing on several gazelle websites.

Backend: INA

  • Support videos on www.ina.fr (in addition to boutique.ina.fr).

Backend: IPInfoDB

  • Fix: no crash when there are no coordinates from server.

Backend: Kickass

  • Fix: parsing of search results (#663).

Backend: MediaWiki

  • Handle errors from API.

Backend: MeteoFrance

  • Fix: website changes.

Backend: Orange

  • Fix: handling of pages.

Backend: PirateBay

  • Fix: parsing of search results.

Backend: Redmine

Backend: Transilien

  • Supports the roadmap feature (#681).

Backend: Youtube

  • Support more URLs.
  • Support authentication.
  • Fix: website changes.

Application: boobank

  • Add the 'qif' formatter.

Application: boobank-munin

  • When handling an incorrect password error, disable backend to prevent your bank to disable your account (for example with BNP).

Application: pastoob

  • Ability to pipe output.

Application: traveloob

  • New command 'roadmap' to get the roadmap between to stations. It's also possible to give a departure or an arrival time (#681).

Application: videoob

  • Do not run player in background (#664).
  • Command 'download' supports mms streams.

Application: weboorrents

  • The given filename to command 'getfile' is optional (#683).

Weboob 0.8

Added by Romain Bignon 9 months ago

Weboob 0.8 has been released this week-end, followed by Weboob 0.8.1 which fixes several minor bugs.

This version comes with important changes in the project. Firstly, Weboob has changed its license, from GPLv3 to AGPLv3+.

Then, the release management has also changed. Now, instead of releasing a new version every month, a minor version (x.y) comes out from the unstable git repository (weboob.git) where are merged new features, backends and applications. In parallel, the stable git repository (weboob-stable.git) merges only bugfixes, and a new bugfix version (x.y.z) is released each time it's necessary. A major version (x.0) is released when there are important changes in the architecture, or when a critical API is broken.
The interest of this new management is that bugfixes will come faster, because when a backend is broken, a new bugfix version is immediately released. Also, about minor versions, it isn't necessary to frequently release regardless of importance of new features or stability.

Finally, the third thing is that we have organized the first boobathon! A Boobathon is an event with Weboob developers or contributors, where the goal is to write the more backends possible!

In April, participants were Romain, Laurent, Johann, Christophe and Noé. The winner, who has written successfully two backends, was Johann.

Weboob 0.8 introduces several new features, like collections, several bug fixes, support of galleries and pastes, and also 5 backends:


E-Hentai
France Inter
LeMouv
PasteALaCon
PasteBin

There are now 42 backends and 23 applications.

Credits

ChangeLog

General

Repl applications

  • The new CapCollection capability is supported by every REPL Application. At the moment, only few backends implement it.
  • Better messages and return codes in applications.

Backend: AuM

Backend: BNPorc

  • The 'rotating_password' parameter is now an hidden one (#627).
  • Ability to do transfers to external recipients.

Backend: BP

  • Add 'comptes titres', 'comptes vies' and 'comptes retraites' to the list of accounts (#567).
  • Fix: website changes (#621).

Backend: CanalPlus

Backend: CrAgr

  • Support of a new history layout.
  • Fix: login issues with the Toulouse website (#629).

Backend: Dailymotion

  • Fix: URL was not found in special cases.

Backend: FourChan

  • Fix: support message "AMERICA FUCK YEAH" outside of a thread.

Backend: Gazelle

  • Fix: work with broken gazelle websites.

Backend: Inrocks

  • Fix: matching URLs.

Backend: KickAss

  • Several fixes and enhancements (#651).

Backend: LCL

  • Fix: websites changes.

Backend: OuiFM

Backend: Transilien

  • Add the PAA code for Gare de Lyon.

Backend: Youtube

  • Support more URLs.

Backend: Youporn

  • Correctly set the 'ext' attribute on videos.

Application: monboob

  • Check configuration (#484).

Application: weboob-config

  • New commands 'enable' and 'disable'.

Core

  • Use lowercase http_proxy environment variable.
  • select() function has been moved into parser.
  • Support for xpath in LxmlHtmlParser.select.
  • Fragments are removed from URLs (#636).
  • Remove a hack from feedparser which fixes parsing of webpages with recent versions of this library.
  • Also log redirects when saving responses and debugging (#398).

Weboob 0.7

Added by Romain Bignon 10 months ago

Weboob 0.7 is today out. It introduces mainly several bugfixes, but also new features and two new backends:


Dailymotion
Ecrans

There are now 37 backends and 20 applications.

Credits

ChangeLog

General

Console applications

  • Do not print escape characters (like bold) when using -O.

Backend: AuM

  • Fix: parsing of smileys (#532).
  • Fix: new page (#535).

Backend: BP

  • Fix: parsing of accounts is case there are missing sections.

Backend: CreditMutuel

  • Compatibility with other agencies (#550).

Backend: DLFP

  • Ability to plusse/moinse contents.
  • Ability to tag a content.
  • Support the board.
  • Support comments signatures.
  • Support wiki, forums, polls, tracker.
  • Now implements CapContent to edit wiki.

Backend: FourChan

  • All messages are children of thread.

Backend: Inrocks

  • Add support of 'InrocksTV' pages.

Backend: IpInfoDB

  • Fix: crash when lat/long are empty (#585).

Backend: LCL

  • Fix login, and implement CapBank methods.

Backend: Newsfeed

  • Fix: correctly set the UNREAD flag.

Backend: Orange

  • Fix: posting messages (#614).

Backend: Redmine

  • Fix: support project names with '-' in.

Backend: Transilien

  • Fix: don't keep previous results.
  • Change user-agent of browser and use HTTPS (#580,#583).

Backend: Youjizz

  • Fix: crash when duration is not available.

Backend: Youtube

  • Now correctly handle gdata errors.
  • Fix: get video from page URL.
  • Fix: get file URL of video.

Application: boobmsg

  • New command 'export_all'.
  • New xhtml formatter.

Application: masstransit

  • Works without conic or hildon (#562).
  • Add banner on load.

Application: QWebContentEdit

  • Better errors management.

Application: videoob

  • New command 'download'.

Application: webcontentedit

  • Set a default editor (#557).

Core

  • Split of ReplApplication to create ConsoleApplication.
  • Remove the 'refresh' feature from Browser (to prevent unwanted behaviors like freezes).
  • Browser has upgraded Firefox from 3.0.3 to 3.6.13.
  • Browser doesn't retry on 404 errors.
  • Script to generate Sphinx API documentation.

Weboob 0.6

Added by Romain Bignon 11 months ago

With a month late, we are happy to announce this new version of Weboob!

There are mostly new backends and several bugfixes:


Inrocks
Le Figaro
MediaWiki
20 minutes

And also a new great Qt application:


QWebContentEdit

There are now 35 backends and 20 applications.

Credits

ChangeLog

General

Backend: AuM

  • Fix: website changed, mails.php renamed to mail.php.

Backend: BNPorc

  • Fix: the captcha has been changed on their website (#461).

Backend: CrAgr

  • Fix: the history operation works when every non-ASCII character on
    the bank website is replaced with two interrogation marks.
  • Fix: calculation of cents.

Backend: CreditMutuel

  • Fix: negative value correctly considered by 'list'

Backend: DLFP

  • Fix: website changed to a new version (#503).

Backend: IsoHunt

  • Fix: website doesn't provide seed information on info page anymore (#529).

Backend: KickAss

  • Fix: size span detection improved (fixes #497).

Backend: OuiFM

  • Fix: website changed to get current song.

Backend: PirateBay

  • Fix: parsing to find leecher and seeders (#458).

Backend: Redmine

  • Adds support for previewing modifications on redmine's wiki pages.
  • Fix: when login/password is invalid.

Backend: Yahoo

  • Fix: didn't load with python < 2.6

Application: boobmsg

  • New command 'export_thread'.
  • New parameter '-t' on the 'post' command to set a title.

Application: monboob

  • Fix: catch errors when trying to bind the SMTP server (#485).

Application: QBoobMsg

  • Fix: reload of backends list.

Application: webcontentedit

  • New command 'log' to display all revisions of a page.

Application: weboob-config

  • The 'edit' command can be used to interactively edit one backend,
    instead of opening the './weboob/backends' file with a text editor.
  • Checks on given backends for 'backends <add|register>' subcommands.

Application: weboob-config-qt

  • Fix: crash when trying to add an already existing backend.

Repl Applications

  • New parameter '-O' to set an output file.

Core:

  • Fix: Browser.location() crashes when uri is empty (#488).
  • Fix: catch every exceptions when loading a backend (not only ImportError).

Weboob 0.5

Added by Romain Bignon about 1 year ago

This new version of Weboob comes with only minor new features, several bugfixes, and two new backends:


MeteoFrance
Orange

There are now 31 backends and 19 applications.

Credits

ChangeLog

General

Backend: Bouygues

  • Fix: logging issues (#444).

Backend: CrAgr

  • Fix: do not keep accounts list in cache.
  • Fix: extraction of amounts >999€ and negative amounts.

Backend: KickAss

  • Fix: website changes (#457).
  • Fix: size parsing.

Backend: PirateBay

  • Fix: size parsing.

Backend: Yahoo

  • The 'search' command is implemented.

Backend: Youtube

  • Fix: website changes (#450).

Application: boobmsg

  • The 'post' command can take message from command-line instead of stdin.
  • Fix: encoding problems.

Application: videoob

  • Fix: playing videos in some cases (#446).

Application: wetboobs

  • Rewrite of application to use formatters.

Weboob 0.4 (6 comments)

Added by Romain Bignon about 1 year ago

I'm happy to announce that Weboob 0.4 is out.

Nine backends have been added:


Bouygues
Canal Plus
Crédit Mutuel

IPInfoDB
IsoHunt
KickAss

Pirate Bay
SFR
Société Générale

And a new application has been created:


boobmsg

There are now 29 backends and 19 applications!

This release introduces also lot of fixes for previous backends, a new and powerful optimization management for the AuM, and several other features…

With this release, the new version of Weboob 0.3 is out: Weboob 0.3.1, which fixes several bugs.

Credits

ChangeLog

General

Console applications

  • New command inspect to open a graphical webkit browser with the current page (to help debugging). If it is not available, the page source is displayed on stdout.
  • In question prompts, display a bullet list when there are too many choices.
  • The --save-responses (-a) parameter stores now pages in a directory and save here a debug log and a list of visited URLs associated to the files.
  • Fix unicode issues (#436).

Backend: AuM

  • Do not send baskets messages from sluts.
  • Anti-spam is updated.
  • Raise an error when user is banned.
  • New optimization PRIORITY_CONNECTION to create fake godchilds to allow user access to website between 18h and 1h. (#319)
  • New optimization QUERIES_QUEUE to enqueue queries in case no charms are available.
  • New backend parameter 'baskets' to enable getting baskets messages.
  • In profiles, look for hidden photos.

Backend: BNPorc

  • Fix: display of coming operations.
  • Fix: check if the password rotation has been succeed.

Backend: BP

  • Backend has been rewritten to use BaseBrowser (#422).
  • Fix: backend initialization.
  • Fix: parsing of accounts.
  • Fix: handle of transfers errors.

Backend: CrAgr

  • Support of history operations.
  • Support monay transfers.
  • Choose the agency website in a list instead of giving URL.

Backend: DLFP

  • Store datetime from newsfeed in the Thread object (#419).
  • Now the session is closed at deinit.
  • Fix: when posting a comment, send right parameters to immediately display it.

Backend: GeolocIP

Backend: INA

  • Fix: parsing date, duration and title.

Application: boobank-munin

  • Save cache in .weboob/munin/ and handle errors.
  • New option 'boobank_cumulate' to display graph as areas instead of plots.

Application: havesex

  • The optimizations management has been rewritten.
  • New command 'query' to send a query to a contact (like a charm or a poke).
  • Fix: do not exist after displaying a profile in interactive mode.

Application: monboob

  • New option to pipe mails to an external process instead of sending it to a SMTP server.
  • Fix: when domain in In-Reply-To contains a '@'.
  • Fix: parsing incoming mails when no charset is supplied.
  • Fix: unicode issues.

Application: QHaveSex

  • Display URL of contacts.
  • Contacts list is now sorted.
  • Have a photos caroussel on profile page.

Application: weboob-config

  • New command confirm. It takes an email in stdin and call corresponding backend to go on the confirm address. It is useful to automatically confirm account register.

Application: weboorrents

  • Ability to complete paths.

Core

  • The 'repeat' scheduler has been rewritten.
  • Ability to cancel a scheduled task.
  • Fix parsing of path-like in INI config.
  • Conditions are now treated by BackendsCall instead by formatters (#372).
  • Backends name can now contain only letters and digits.
  • Add a tool to generate manpages.

Weboob 0.3

Added by Romain Bignon over 1 year ago

This third version of Weboob introduces lot a changes. There are several changes in backends, and REPL applications have been enhanced.

Then, four backends have been added:


LCL
Newsfeed
OuiFM
Redmine

Also, two applications have been created:


radioob
webcontentedit

ChangeLog:

General

Console Applications

  • Formatters management have been rewritten. Now each command can set its own default formatter, and user can redefine them.
  • If output exceed the height of term, ask user to press a key for each page.
  • Do not display columns when all of these values are NotLoaded or NotAvailable.
  • Add a CSV formatter (#350).
  • Command 'backends register' to register a new account (#294).
  • Can use '$full' and '$direct' selectors.

Backend: Arte

  • Fix: fall-back when the wanted quality is not available.

Backend: AuM

  • New anti-spam feature to detect, block and report spambots (#313).
  • Implements the capability CapAccount to register new accounts (#389).
  • In profile walker, when reloading sluts list from search page, do not keep the previous queue to prevent visiting sluts who have been added a long time before and who are now disconnected.
  • Contact status is now 'connected'/'not connected since ... hours'.
  • Fix: do not crash if contact list is empty.

Backend: BNPorc

  • If password is expired, switch with the value of the new 'rotating_password' backend setting.
  • Support transfers (#416).
  • Fix: don't crash when accounts don't support 'coming' (#401).
  • Fix: when website is not available, raise BrowserUnavailable instead of BrowserIncorrectPassword.

Backend: DLFP

  • Replace 'cite>' and 'tt>' to 'i>' in read messages.

Application: boobank

  • Use 'table' as default formatter.
  • Output for the 'list' command is more readable (#410).
  • When a transfer is made, display all information instead of ID.
  • Fix: do not load backends twice (#405).

Application: QBoobMsg

  • Support threads display and answers.
  • Support sending answers (plaintext or HTML).
  • Unread messages are in yellow in list, and set message as read when one is selected.
  • Fix: correctly reload when backends have been changed.

Application: QHaveSex

  • Ability to display a profile from URL.

Application: QVideoob

  • Fix: search on youtube was unlimited, so QVideoob freezed.

Application: traveloob

  • Use 'table' as default formatter.

Application: videoob

  • Output for the 'search' command is more readable.

Application: weboob-config

  • Add a 'register' command (#294).

Application: weboob-config-qt

  • Add a 'Register' button to register a new account on a backend (#390).

Application: weboorrents

  • Output for the 'search' and 'info' commands is more readable.

Core

  • Weboob.load_backends() takes a new 'errors' argument.
  • CapBaseObject has a new method 'add_field' to add a field property which forces a specific type and is set to NotLoaded by default.
  • Browser.readurl() can take same arguments than Browser.openurl().
  • If a page is not recognized by the Browser instance, save response even if -a is not supplied.
  • Introduce 'Value' classes to replace BaseBackend.ConfigField and is used by ReplApplication.ask().
  • Use several loggers for parts of weboob.
  • Fix: crash when HTTP server returns shit (#406).

Weboob 0.2 (1 comment)

Added by Romain Bignon over 1 year ago

This new version of Weboob introduces major changes.

Firstly, every console applications have now a REPL mode (interactive), which is intuitive and stateful, and supports completion.

Then, three new backends have been added:


Arte
Banque Postale
GeolocIP

Also, an automated tests architecture has been written and setup, to prevent regressions and websites changes which would make some backends broken.

ChangeLog:

Links:

Credits:

  • Romain Bignon
  • Christophe Benz
  • Nicolas Duhamel
  • Laurent Bachelier
  • Flo

Weboob 0.1

Added by Romain Bignon over 1 year ago

Today weboob is out for the first time. This project started six months ago, and after 837 commits is now enough stable to be publicly released.

Currently, there are ten capabilities: 13 backends:

And 15 applications:

Core team:
  • Romain Bignon
  • Christophe Cbenz
Contributors:
  • Laurent Bachelier
  • Roger Philibert
  • Julien Hébert

Have fun!

Weboob at LSM/RMLL 2010

Added by Romain Bignon over 1 year ago

Weboob is a new project with the goal to provide several clients to interact with a lot of websites.

Currently, there are 13 backends (about bank, chat, contacts management, dating, messages, torrents, travel, video and weather), and 17 applications (daemons, console, qt, n900).

The Libre Software Meeting is a free (as a beer and as a speech) and non commercial conference with talks, workshops and round tables about Free Software and its uses.
A lecture about Weboob will take place on Thursday 8th July, at 3.40pm.

Also available in: Atom