Bug #1820

[citibank] Make scraping more robust

Added by Oleg Plakhotniuk about 2 years ago. Updated about 2 years ago.

Status:Resolved Start:2015-03-31
Priority:Normal Due date:
Assigned to:Oleg Plakhotniuk % Done:

100%

Category:Modules Spent time: -
Target version:-
Module:citibank Branch:

Description

There are a few errors occuring sporadically. I'll keep this issue around for a while and accumulate my fixes here.

Associated revisions

Revision dbf13930a5c28764a82a4b816b8fa69845ce2f83
Added by Oleg Plakhotniuk about 2 years ago

[citibank] Relogin when the website returns garbage instead of a PDF. Closes #1820

History

Updated by Oleg Plakhotniuk about 2 years ago

  • Status changed from In progress to To merge
  • % Done changed from 0 to 100

I think it's good to merge. The fix has been working for a week now without any issues.

Done in branch issue1820

Updated by Oleg Plakhotniuk about 2 years ago

  • Status changed from To merge to In progress
  • % Done changed from 100 to 50

Site got updated and now throws an exception. This issue's branch still isn't merged, so I'll just add the fix here.

Exception in question:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-1-987d0babe4bb> in <module>()
----> 1 next(backend.iter_accounts())

/srv/pub/projects/weboob/repo/modules/citibank/browser.py in iter_accounts(self)
    189
    190     def iter_accounts(self):
--> 191         for innerId in self.to_accounts().inner_ids_dict().values():
    192             yield self.to_account(innerId).account()
    193

/srv/pub/projects/weboob/repo/modules/citibank/browser.py in inner_ids_dict(self)
     69     def inner_ids_dict(self):
     70         return dict((prod['parsedAccountName'][-4:], prod['accountInstanceId'])
---> 71             for cat in self.doc['accountsSummaryViewObj']['categoryList']
     72             for prod in cat['products'] if cat['categoryType'] == 'CRD')
     73

KeyError: 'accountsSummaryViewObj'

Updated by Oleg Plakhotniuk about 2 years ago

  • Status changed from In progress to To merge
  • % Done changed from 50 to 100

Updated by Oleg Plakhotniuk about 2 years ago

  • Status changed from To merge to Resolved

Also available in: Atom PDF