Bug 408284 - Set CSV import format per account
Summary: Set CSV import format per account
Status: RESOLVED FIXED
Alias: None
Product: skrooge
Classification: Applications
Component: general (other bugs)
Version First Reported In: unspecified
Platform: Other Linux
: NOR wishlist
Target Milestone: 25.1.0
Assignee: Stephane MANKOWSKI
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-06-04 11:06 UTC by Dotan Cohen
Modified: 2020-03-16 21:51 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed/Implemented In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dotan Cohen 2019-06-04 11:06:58 UTC
SUMMARY

Each bank uses a different CSV format, including different Date formats. I need to configure Skrooge to understand each of these formats. Preprocessing the files with Python is not an option as it adds a significant step in difficulty for the user.
Comment 1 Stephane MANKOWSKI 2019-06-04 18:43:27 UTC
Hi,

The automatic detection of headers of CSV files is done for that.
By setting, the right regular expressions, you should be able to detect automatically different formats of CSV files.
Comment 2 Dotan Cohen 2019-06-05 10:58:12 UTC
Thank you Stephane. In the case where header detection is enough, then I agree with you.

However, some CSV files do not have headers. Some CSV files use YYYMMDD date formats and others use DD/MM/YYYY. Some accounts have debits and credits in separate columns, others use a combined column. Many CSV files do not contain the account number.

In my limited experience, a single CSV import format is unable to account for all these possibilities.
Comment 3 Syfer Polski 2019-09-28 16:04:31 UTC
Would it be possible to add a manually selectable parser for the DD/MM/YYYY format? HSBC's midata contains the date in this format and trying to import it into Skrooge results in a "Date format not supported" error.

I can provide an example file if that would help. The headers are present, but don't specify the data formats, and the list of transactions is followed by an empty line and an arranged overdraft:

 Date,Type,Merchant/Description,Debit/Credit,Balance
28/08/2019,))),TESCO           LONDON,-£2.00,+£998.75

Arranged overdraft limit,28/09/2019,+£0.00
Comment 4 Stephane MANKOWSKI 2019-09-29 13:52:43 UTC
(In reply to Syfer Polski from comment #3)
> Would it be possible to add a manually selectable parser for the DD/MM/YYYY
> format? HSBC's midata contains the date in this format and trying to import
> it into Skrooge results in a "Date format not supported" error.
> 
> I can provide an example file if that would help. The headers are present,
> but don't specify the data formats, and the list of transactions is followed
> by an empty line and an arranged overdraft:
> 
>  Date,Type,Merchant/Description,Debit/Credit,Balance
> 28/08/2019,))),TESCO           LONDON,-£2.00,+£998.75
> 
> Arranged overdraft limit,28/09/2019,+£0.00

Skrooge ties to automatically find the date format by analyzing the different dates.
The problem is that the last line "Arranged overdraft limit,28/09/2019,+£0.00" is analyzed too and Skrooge doesn't find a date in "Arranged overdraft limit".

Could you try to remove this line and import again?
Comment 5 Stephane MANKOWSKI 2019-10-02 07:43:13 UTC
(In reply to Syfer Polski from comment #3)
> Would it be possible to add a manually selectable parser for the DD/MM/YYYY
> format? HSBC's midata contains the date in this format and trying to import
> it into Skrooge results in a "Date format not supported" error.
> 
> I can provide an example file if that would help. The headers are present,
> but don't specify the data formats, and the list of transactions is followed
> by an empty line and an arranged overdraft:
> 
>  Date,Type,Merchant/Description,Debit/Credit,Balance
> 28/08/2019,))),TESCO           LONDON,-£2.00,+£998.75
> 
> Arranged overdraft limit,28/09/2019,+£0.00

Hi,

I did a modification to support CSV file with footer.
https://cgit.kde.org/skrooge.git/commit/?id=cb5bfce23fc18fcf767adba3219a591370b5ac7e

Anyway, if you remove the line "Arranged overdraft limit,28/09/2019,+£0.00", import will work.
Comment 6 Syfer Polski 2019-10-03 21:23:15 UTC
Thank you Stephane!