Cannot get stock prices. Returns error: --------------------------------------------------------------------------- Fetching URL http://funds.ft.com/uk/Tearsheet/Summary?s=GB00074476255... Unable to update price for GB000744762 (no price or no date) --------------------------------------------------------------------------- Same error for every other stock price. Have tried other sources (Yahoo, Yahoo UK, etc.) & get same error. System is running Manjaro 7.0.1. I've tried on a second system with same update level & get the same problem. This was working until a week ago. Not clear if caused by Manjaro update or new KMyMoney version.
FT.com have changed that page layout recently. I got it working by changing the KMyMoney settings to these. URL: http://funds.ft.com/uk/Tearsheet/Summary?s=%1:GBX Symbol: data-display-symbol="(.*): Price: Price \(GBX\)</span><span class="mod-ui-data-list__value">([\d,]*\.\d*)</span> Date: Data delayed at least 15 minutes, as of (.*) Date Format: %m %d %y Hope that helps. David On Monday 19 June 2017 14:38:11 StephenJH wrote: > https://bugs.kde.org/show_bug.cgi?id=381399 > > Bug ID: 381399 > Summary: Can't retrieve stock prices > Product: kmymoney4 > Version: 4.8.0 > Platform: Other > OS: Linux > Status: UNCONFIRMED > Severity: normal > Priority: NOR > Component: onlinebanking > Assignee: kmymoney-devel@kde.org > Reporter: stephen.henstridge@gmail.com > Target Milestone: --- > > Cannot get stock prices. Returns error: > > --------------------------------------------------------------------------- > Fetching URL http://funds.ft.com/uk/Tearsheet/Summary?s=GB00074476255... > Unable to update price for GB000744762 (no price or no date) > --------------------------------------------------------------------------- > > Same error for every other stock price. > > Have tried other sources (Yahoo, Yahoo UK, etc.) & get same error. > > System is running Manjaro 7.0.1. I've tried on a second system with same update > level & get the same problem. > > This was working until a week ago. Not clear if caused by Manjaro update or new > KMyMoney version. > >
David, Thanks for the reply. I made the changes. It now works for about 9 out of 10 of the different requests I make. The others return a date but no value. I wonder if there is some variation between the different HTML pages. Anyway, thanks again. I'll keep trying it to see if it recovers. cheers, Stephen.
(In reply to David Houlden from comment #1) > FT.com have changed that page layout recently. I got it working by changing > the KMyMoney settings to these. > > URL: http://funds.ft.com/uk/Tearsheet/Summary?s=%1:GBX ... > Price: Price \(GBX\)</span><span > class="mod-ui-data-list__value">([\d,]*\.\d*)</span> How should this work for non GBX currencies ? The initial url uses http://funds.ft.com/uk/Tearsheet/Summary?s=%1 and the website redirects to the currency related result e.g. if I enter http://funds.ft.com/uk/Tearsheet/Summary?s=AT0000639083 I get https://markets.ft.com/data/funds/tearsheet/summary?s=AT0000639083:EUR.
(In reply to StephenJH from comment #0) > Fetching URL http://funds.ft.com/uk/Tearsheet/Summary?s=GB00074476255 This looks to be too unspecific because entering this url in a browser a list of 100 funds is returned. I instead if I use for example http://funds.ft.com/uk/Tearsheet/Summary?s=AT0000639083 in the browser it shows only one stock, which is parsable by kmymoney.
(In reply to David Houlden from comment #1) > FT.com have changed that page layout recently. I got it working by changing > the KMyMoney settings to these. > > URL: http://funds.ft.com/uk/Tearsheet/Summary?s=%1:GBX > Symbol: data-display-symbol="(.*): > Price: Price \(GBX\)</span><span > class="mod-ui-data-list__value">([\d,]*\.\d*)</span> > Date: Data delayed at least 15 minutes, as of (.*) > Date Format: %m %d %y As far as I can see looks the following regex be more general: URL: http://funds.ft.com/uk/Tearsheet/Summary?s=%1 Symbol: data-display-symbol="(.*): Price: class="mod-ui-data-list__value">([\d,]*\.\d*)</span> Date: Data delayed at least 15 minutes, as of (.*)\. Date Format: %m %d %y Changes are in Price: and Date:
Ralph, I tried your modified settings and they worked perfectly, including for a couple of stocks that had never worked before! Many thanks. On the whole, this method of retrieving data (screen scraping) seems very unreliable. I'm surprised there is not Web Service for this. Thanks again. Regards, Stephen. (In reply to Ralf Habacker from comment #5) > (In reply to David Houlden from comment #1) > > FT.com have changed that page layout recently. I got it working by changing > > the KMyMoney settings to these. > > > > URL: http://funds.ft.com/uk/Tearsheet/Summary?s=%1:GBX > > Symbol: data-display-symbol="(.*): > > Price: Price \(GBX\)</span><span > > class="mod-ui-data-list__value">([\d,]*\.\d*)</span> > > Date: Data delayed at least 15 minutes, as of (.*) > > Date Format: %m %d %y > > As far as I can see looks the following regex be more general: > > URL: http://funds.ft.com/uk/Tearsheet/Summary?s=%1 > Symbol: data-display-symbol="(.*): > Price: class="mod-ui-data-list__value">([\d,]*\.\d*)</span> > Date: Data delayed at least 15 minutes, as of (.*)\. > Date Format: %m %d %y > > Changes are in Price: and Date:
Sorry, Ralf, I spelled your name wrong :-(
BTW: There is bug showing fetched web quotes, see 381419
(In reply to StephenJH from comment #6) > On the whole, this method of retrieving data (screen scraping) seems very > unreliable. > I'm surprised there is not Web Service for this. https://stackoverflow.com/questions/3541393/stock-and-mutual-fund-data-web-service mentions some services
(In reply to StephenJH from comment #6) > > Symbol: data-display-symbol="(.*): This seems also to be outdated, I can't find the related term on the result page https://markets.ft.com/data/funds/tearsheet/summary?s=LU0063916489:USD. The related part on the page looks to be: div class="mod-tearsheet-overview__header__symbol"><span>LU0063916489:USD</span> which may be parsed with the following regex: Symbol: class="mod-tearsheet-overview__header__symbol"><span>([a-zA-Z0-9:]+)</span> the updated record is then: URL: http://funds.ft.com/uk/Tearsheet/Summary?s=%1 Symbol: class="mod-tearsheet-overview__header__symbol"><span>([a-zA-Z0-9:]+)</span> Price: class="mod-ui-data-list__value">([\d,]*\.\d*)</span> Date: Data delayed at least 15 minutes, as of (.*)\. Date Format: %m %d %y
(In reply to Ralf Habacker from comment #10) > (In reply to StephenJH from comment #6) > > > Symbol: data-display-symbol="(.*): > > This seems also to be outdated, I can't find the related term on the result > page https://markets.ft.com/data/funds/tearsheet/summary?s=LU0063916489:USD. > > The related part on the page looks to be: > > div > class="mod-tearsheet-overview__header__symbol"><span>LU0063916489:USD</span> > > which may be parsed with the following regex: > > Symbol: > class="mod-tearsheet-overview__header__symbol"><span>([a-zA-Z0-9:]+)</span> > > the updated record is then: > > URL: http://funds.ft.com/uk/Tearsheet/Summary?s=%1 > Symbol: > class="mod-tearsheet-overview__header__symbol"><span>([a-zA-Z0-9:]+)</span> > Price: class="mod-ui-data-list__value">([\d,]*\.\d*)</span> > Date: Data delayed at least 15 minutes, as of (.*)\. > Date Format: %m %d %y Hi Ralf There are more outdated Quote Sources and I prepared patch for them which is available on https://phabricator.kde.org/D6224 I see you're not on phabricator and using old ReviewBoard. Please consider switching, so all development occurs in one place. funds.ft.com were working at time of writing the patch, but I'll check it anew and would want to include any new changes into it. Regards Łukasz
(In reply to Ralf Habacker from comment #8) > BTW: There is bug showing fetched web quotes, see 381419 StephenJH points out that this is a stock account configuration issue and does not block this bug.
Git commit dd770434613dab18810e30de114e7541abd8861a by Ralf Habacker. Committed on 20/06/2017 at 08:08. Pushed by habacker into branch '4.8'. Make debug output of parsed fields from online web price quotes more visible by wrapping with '. M +3 -3 kmymoney/converter/webpricequote.cpp https://commits.kde.org/kmymoney/dd770434613dab18810e30de114e7541abd8861a
(In reply to Ralf Habacker from comment #3) > (In reply to David Houlden from comment #1) > > FT.com have changed that page layout recently. I got it working by changing > > the KMyMoney settings to these. > > > > URL: http://funds.ft.com/uk/Tearsheet/Summary?s=%1:GBX > ... > > Price: Price \(GBX\)</span><span > > class="mod-ui-data-list__value">([\d,]*\.\d*)</span> > > How should this work for non GBX currencies ? > > The initial url uses http://funds.ft.com/uk/Tearsheet/Summary?s=%1 > > and the website redirects to the currency related result e.g. if I enter > > http://funds.ft.com/uk/Tearsheet/Summary?s=AT0000639083 > > I get > > https://markets.ft.com/data/funds/tearsheet/summary?s=AT0000639083:EUR. You are correct Ralf, this was just my fix which works for me. I added the GBX for a good reason a long time ago and I am not sure if I need it anymore. In the past, that web site has changed the investment prices from GBX to GBP and back again on some investments. If the factor (number of decimal places) in KMyMoney is set to assume the price will be in GBX and they change to GBP then the stored price is wrong. Therefore I added GBX to the regexp so I got an error if ft,com changed the price units.
David, I've had the same experience with prices changing dramatically. I find it easier to just check that the returned price makes sense and then adjust the scaling factor for the stock if necessary. Stephen. (In reply to David Houlden from comment #14) > (In reply to Ralf Habacker from comment #3) > > (In reply to David Houlden from comment #1) > > > FT.com have changed that page layout recently. I got it working by changing > > > the KMyMoney settings to these. > > > > > > URL: http://funds.ft.com/uk/Tearsheet/Summary?s=%1:GBX > > ... > > > Price: Price \(GBX\)</span><span > > > class="mod-ui-data-list__value">([\d,]*\.\d*)</span> > > > > How should this work for non GBX currencies ? > > > > The initial url uses http://funds.ft.com/uk/Tearsheet/Summary?s=%1 > > > > and the website redirects to the currency related result e.g. if I enter > > > > http://funds.ft.com/uk/Tearsheet/Summary?s=AT0000639083 > > > > I get > > > > https://markets.ft.com/data/funds/tearsheet/summary?s=AT0000639083:EUR. > > You are correct Ralf, this was just my fix which works for me. I added the > GBX for a good reason a long time ago and I am not sure if I need it > anymore. In the past, that web site has changed the investment prices from > GBX to GBP and back again on some investments. If the factor (number of > decimal places) in KMyMoney is set to assume the price will be in GBX and > they change to GBP then the stored price is wrong. Therefore I added GBX to > the regexp so I got an error if ft,com changed the price units.
Git commit 618324114b56ae14950f3ab990c718cbc41d3fce by Łukasz Wojniłowicz. Committed on 20/06/2017 at 19:21. Pushed by wojnilowicz into branch 'master'. Update Web Price Quote Sources Some sources were added, some removed (no longer work and cannot be made to work), and some updated. FIXED-IN:5.0 Differential Revision: https://phabricator.kde.org/D6224 Signed-off-by: Łukasz Wojniłowicz <lukasz.wojnilowicz@gmail.com> M +104 -119 kmymoney/converter/webpricequote.cpp https://commits.kde.org/kmymoney/618324114b56ae14950f3ab990c718cbc41d3fce