Bug 332380 - Easier import of bibliographies
Summary: Easier import of bibliographies
Status: ASSIGNED
Alias: None
Product: KBibTeX
Classification: Applications
Component: General (show other bugs)
Version: 0.5
Platform: openSUSE Linux
: NOR wishlist
Target Milestone: ---
Assignee: Thomas Fischer
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-03-20 22:34 UTC by auxsvr
Modified: 2016-01-13 13:50 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
The patch implementing basic dbus support for kbibtex 0.7 (based on 6d9d476) (40.26 KB, patch)
2015-12-01 00:18 UTC, Shunsuke Shimizu
Details
Declare constants for document and file id numbers. (2.79 KB, patch)
2015-12-04 07:51 UTC, Shunsuke Shimizu
Details
revised python code (2.22 KB, text/x-python)
2015-12-15 15:22 UTC, Shunsuke Shimizu
Details

Note You need to log in before you can comment on or make changes to this bug.
Description auxsvr 2014-03-20 22:34:48 UTC
It would be very time-saving if kbibtex supported either a command-line option or a dbus interface to add a bibliography record to the currently open file. This is important, because most article databases either offer a .bib file with the bibliography or display the text, which one then needs to copy & paste into kbibtex. For example, with scripting I could make klipper import the bibliography upon selecting the text, requiring only one click. 

Also, it would be nice if there was an option to import the bibliographies in the new file, if one clicks on it and another file is already open.

Reproducible: Always
Comment 1 Thomas Fischer 2014-03-24 21:49:49 UTC
As I don't use DBus on such a level, but you seem to have a good idea, can you please describe in more detail how such a DBus interface should look like? Is there another KDE application which implements a similar feature so that I can see how this application has realized the interface?
Comment 2 Thomas Fischer 2014-10-14 18:43:15 UTC
(In reply to Thomas Fischer from comment #1)
> As I don't use DBus on such a level, but you seem to have a good idea, can
> you please describe in more detail how such a DBus interface should look
> like? Is there another KDE application which implements a similar feature so
> that I can see how this application has realized the interface?
auxsvr, can you please provide the information I requested in comment #1?
Comment 3 auxsvr 2014-10-15 11:22:36 UTC
I was thinking about something like a combination of kate's -i command line option and the URL argument, which would copy the bibtex code from the URL into a new entry in the already opened bibliography. I haven't been able to find something more sophisticated using dbus and I'm not very experienced in C++/Qt programming to give more information about the technical details this would entail.
Comment 4 Christoph Feck 2014-11-16 16:27:54 UTC
Thomas, does comment #3 provide the requested information? Please set the bug status or add a comment.
Comment 5 Thomas Fischer 2014-12-25 21:02:30 UTC
(In reply to Christoph Feck from comment #4)
> Thomas, does comment #3 provide the requested information? Please set the
> bug status or add a comment.

I am working currently on some code for this feature. A HTML importer already exists, so you are able to open a .html file and KBibTeX will try to detect and parse any BibTeX code in this HTML file.
I also started on the -i and DBus support by re-using Kate's code. You can pass argument -i to KBibTeX, it will read data from stdin, but send it to Kate for now (just to demonstrate that it is working).
The code is available in my personal clone at:
http://commits.kde.org/clones/kbibtex/thomasfischer/kbibtex/8c909753aeb3af7e74bf2abec8d885cb7e2fdcd0
Comment 6 Christoph Feck 2015-01-14 23:50:14 UTC
Thanks for the update, setting status correctly.
Comment 7 Thomas Fischer 2015-02-01 21:26:38 UTC
I removed the whole "reading from command line", as it did not work properly.

What is working now is that KBibTeX will expose a DBus method "insertBibEntry" (two arguments: text fragment with bibliography and mime type (only "text/x-bibtex" for now)).
You can send BibTeX fragments to KBibTeX like this (change servicename):

qdbus org.gna.kbibtex-3132 /MainApplication org.gna.KBibTeX.Application.insertBibEntry '@article{aaa,title="No title",year="2000"}' 'text/x-bibtex'

This code is available as of commit 6326c7405053:
http://commits.kde.org/clones/kbibtex/thomasfischer/kbibtex/6326c74050535fa48412f1e38320579f75d68674
Comment 8 auxsvr 2015-02-09 16:17:37 UTC
Thanks, I have cloned the branch bugs/kde332380 and the dbus method works, indeed. However, kbibtex is not notified that the opened file has changed.
Comment 9 Shunsuke Shimizu 2015-12-01 00:18:04 UTC
Created attachment 95830 [details]
The patch implementing basic dbus support for kbibtex 0.7 (based on 6d9d476)

Dear auxsvr, Thomas,

It seems the "correct" way to implement dbus functionality is, as dbus object, publishing each opened bibtex document (tied to KBibTeXPart) and files on document lists.
I tried that and succeed to implement basic functionalities: listing of opened files, opening another file, and insertion of elements. Though currently entry editing operation is limited, it is also a start point to make KBibTeX fully controlable from other applications.

Following python scripts demonstrates the functionalities. It create "/tmp/foo.bib" with two entries. 

import dbus
import re
import sys

bus = dbus.SessionBus()

names = bus.list_names()
kbibtex_names = filter(lambda name: re.match(r"\Aorg.gna.kbibtex-\d+\Z", name), names)
if not kbibtex_names:
	print("kbibtex is not running")
	sys.exit(1)
kbibtex_name = kbibtex_names[0]
print(kbibtex_name)

file_manager = bus.get_object(kbibtex_name, '/FileManager')
file_manager_interface = dbus.Interface(file_manager, dbus_interface = 'org.gna.KBibTeX.FileManager')

opened_id = int(file_manager.open('file:///tmp/foo.bib'))
file_manager.setCurrentFileId(opened_id)
print("opend id: "+str(opened_id))
current_file_id = int(file_manager_interface.currentFileId())
print("current file id: "+str(current_file_id))
if not current_file_id:
	print("no file is opened")
	sys.exit(2)

current_file = bus.get_object(kbibtex_name, '/FileManager/'+str(current_file_id))
current_file_interface = dbus.Interface(current_file, dbus_interface = 'org.gna.KBibTeX.FileManager.FileInfo')
current_url = str(current_file_interface.url())
print("current url: "+current_url)
current_document_id = int(current_file_interface.documentId())
print("current document id: "+str(current_document_id))
if not current_document_id:
	print("no document is opend")
	sys.exit(3)

current_document = bus.get_object(kbibtex_name, '/KBibTeX/Documents/'+str(current_document_id))
current_document_interface = dbus.Interface(current_document, dbus_interface = 'org.gna.KBibTeX.Document')
indexes = current_document_interface.insertEntries("""\
@book{Knuth86,
  author    = {Donald E. Knuth},
  title     = {The TeXbook},
  publisher = {Addison-Wesley},
  year      = {1986},
  isbn      = {0-201-13447-0}
}
@book{Lamport86,
  author    = {Leslie Lamport},
  title     = {LaTeX: User's Guide and Reference Manual},
  publisher = {Addison-Wesley},
  year      = {1986},
  isbn      = {0-201-15790-X}
}""", 'application/x-bibtex')
for i in indexes:
	print("entry index: "+str(i))
current_document_interface.documentSave()
Comment 10 Shunsuke Shimizu 2015-12-01 06:58:40 UTC
Another comment.

I'm not sure whether adaptor should be created, KBibTeXPart's one (as does in the patch) or the FileModel's. I make it for the part since it enables the access to GUI, like filter bar, considering users might like to filter their bibs by scripting. Also some functionalities like id generation are easily accessible from the part.

However, such functionalities also can be implemented for FileModel, by separating the processing of FileModel and the use from view, as I did for `clipboard.cpp`. For filtering, we may create a SortFilterFileModel not tied to the filter bar, every time a user send dbus query to obtain filtered indexes with search terms.

Therefore, considering twiddling GUI from external commands is a wrong, now I'm a bit inclined to the adaptation of FileModel. Nevertheless full functionalities need not be exported now. Exporting those is not zero-or-all work, but gradual changing. 

I think those features really helps cooperation with web browsers. The rational for using KBibTeX instead of another individual library is: 
1. Changes will be consistent with KBibTeX's view
2. KBibTeX features plenty of features

Shunsuke
Comment 11 Thomas Fischer 2015-12-03 21:46:44 UTC
(In reply to Shunsuke Shimizu from comment #9)
> Created attachment 95830 [details]
> The patch implementing basic dbus support for kbibtex 0.7 (based on 6d9d476)
Thank you very much for this patch.
I started to test it and made minor adjustments. The Python script is working and confirms the promised functionality.

I split your patch into a number of smaller Git commits, available in my private repository in branch bug/kdepatch95830:
http://commits.kde.org/clones/kbibtex/thomasfischer/kbibtex/393c846ead733a8f7697d65c2c5fdeb6d73e1a1c
Comment 12 Thomas Fischer 2015-12-03 21:48:49 UTC
(In reply to Shunsuke Shimizu from comment #10)
> Another comment.
It is getting late in my time zone. I need to think about this with a fresh mind. I'll come back in a few days.

Thank you again for your work on this bug!
Comment 13 Shunsuke Shimizu 2015-12-04 07:51:57 UTC
Created attachment 95893 [details]
Declare constants for document and file id numbers.

Thank you for your consideration!

Starting the document and file ids from 0 with the invalid id -1 is maybe better. Feel free to revert the change of id numbers. Anyway hardcoding those are vulnerable to changes and mistakes, so I made another small patch to declare those as constants, on the top of your branch (393c846).
Comment 14 Thomas Fischer 2015-12-04 21:31:22 UTC
(In reply to Shunsuke Shimizu from comment #13)
> Starting the document and file ids from 0 with the invalid id -1 is maybe
> better. Feel free to revert the change of id numbers. Anyway hardcoding
> those are vulnerable to changes and mistakes, so I made another small patch
> to declare those as constants, on the top of your branch (393c846).
I applied your patch with minor changes. It will use numbers starting with 0 for valid ids and negative numbers (-1) for invalid ones. Your Python code above had to be updated.

To follow the naming scheme of bug/features branches, I delete the old bugs/kde332380 branch (my own dead attempt to fix this bug) and renamed branch bug/kdepatch95830 to be bugs/kde332380. You may have to delete and/or re-checkout this branch. Sorry for any inconveniences.
Comment 15 Thomas Fischer 2015-12-04 21:37:27 UTC
> I'm not sure whether adaptor should be created, KBibTeXPart's one (as does
> in the patch) or the FileModel's.
Right now, both are working. I suggest to postpone any decision on which one should be removed.

> However, such functionalities also can be implemented for FileModel, by
> separating the processing of FileModel and the use from view, as I did for
> `clipboard.cpp`. For filtering, we may create a SortFilterFileModel not tied
> to the filter bar, every time a user send dbus query to obtain filtered
> indexes with search terms.
I am not sure if I can follow this argumentation. Can you please provide an example?

> I think those features really helps cooperation with web browsers.
Which browsers or browser plugins are using DBus to exchange bibliographies?
Comment 16 Shunsuke Shimizu 2015-12-15 15:14:37 UTC
Thanks a lot for your comments.

(In reply to Thomas Fischer from comment #15)
> > I'm not sure whether adaptor should be created, KBibTeXPart's one (as does
> > in the patch) or the FileModel's.
> Right now, both are working. I suggest to postpone any decision on which one
> should be removed.

I agree.

> > However, such functionalities also can be implemented for FileModel, by
> > separating the processing of FileModel and the use from view, as I did for
> > `clipboard.cpp`. For filtering, we may create a SortFilterFileModel not tied
> > to the filter bar, every time a user send dbus query to obtain filtered
> > indexes with search terms.
> I am not sure if I can follow this argumentation. Can you please provide an
> example? 

Assume someone would like to search documents that contain "foobar" in those title, by using dbus API. There are two ways to provide such functionality:

1. give access to the filter bar and the list of displayed documents
2. provide a function to filter indexes in dbus API. For example, dbus call of `filteredIndexes([1,2,3,4,5],  ["foobar"], AND_SEARCH)` may returns `[2, 5]` (`AND_SEARCH` can be the constant 0 or so).

To implement 2, reusing existing codes to filter documents makes  sense. Since such functionalities are concentrated in the `SortFilterFileModel` class, making an instance of `SortFilterFileModel` per the call of `filteredIndexes` seems to be reasonable way .

> > I think those features really helps cooperation with web browsers.
> Which browsers or browser plugins are using DBus to exchange bibliographies?

Sorry that I said too much. I don't know any such browser or browser plugin. I said that because my primary intention to work on this problem is to automate bibliography importing from firefox, with handmade scripts. I'm using vimperator, so calling such scripts from my browser is fairly easy.
Comment 17 Shunsuke Shimizu 2015-12-15 15:22:01 UTC
Created attachment 96089 [details]
revised python code

> I applied your patch with minor changes. It will use numbers starting with 0 for valid ids and negative numbers (-1) for invalid ones. Your Python code above had to be updated.
I uploaded the new python code. As well the fix of number checking, I also added additional outputs and assertions.
Comment 18 Thomas Fischer 2016-01-05 19:17:39 UTC
I made some changes since early December. Would both of you (auxsvr and  Shunsuke Shimizu) agree that this code is of sufficient quality and feature completeness to be merged into a release branch (e.g. kbibtex/0.6) and this bug to be closed?

http://commits.kde.org/clones/kbibtex/thomasfischer/kbibtex/0f9c0e96b89edd601
Comment 19 Shunsuke Shimizu 2016-01-13 13:50:11 UTC
> I made some changes since early December. Would both of you (auxsvr and 
> Shunsuke Shimizu) agree that this code is of sufficient quality and feature
> completeness to be merged into a release branch (e.g. kbibtex/0.6) and this
> bug to be closed?
> 
> http://commits.kde.org/clones/kbibtex/thomasfischer/kbibtex/0f9c0e96b89edd601

Great! It seems line19 of dbus-example.py `info ("AAA")` is a mistake.
I'm OK for other parts. Thanks.