2013-02-15 Martyn Russell <martyn@lanedo.com>
Release 0.15.2
Merge branch 'fts4-fixes'
2013-02-14 Martyn Russell <martyn@lanedo.com>
ontology: Add a comment that tracker:fulltextNoLimit is now deprecated
Could not mark this as deprecated because it's not a class
libtracker-fts: Fix unit test ontologies, removed tracker:fulltextNoLimit
functional-tests: Removed tracker:fulltextNoLimit
libtracker-data: Removed tracker:fulltextNoLimit
tracker-sparql: Fixed documentation, removed tracker:fulltextNoLimit
ontology: Remove use of tracker:fulltextNoLimit now it's deprecated
2013-02-14 Carlos Garnacho <carlos@lanedo.com>
tests: tweak FTS tests to cope with the word length changes
This has caused some tests to return more results than before,
so add these extra results to the expected output, as the tests
still prove their point.
fts: Remove min word length limit from configuration
It isn't used anymore, so remove the dangling setting.
2013-02-11 Martyn Russell <martyn@lanedo.com>
Merge branch 'fts4'
2013-02-08 Carlos Garnacho <carlos@lanedo.com>
Bump database version
This is needed after the fts4 changes
2013-02-06 Martyn Russell <martyn@lanedo.com>
libtracker-miner: Fixed GNOME password provider using libsecret
- "user" --> "username" for the attributes lookup
- Was checking for SecretItem != NULL to error up the stack
Thanks to Stef Walter for the help here.
2013-02-05 Martyn Russell <martyn@lanedo.com>
tracker-info: Fixed memory leak where urn wasn't freed.
tracker-info: Fixed memory leak when using prefixes for shorthand properties
e.g. nie:url instead of fully qualified URLs
tracker-sparql: Support fts:offsets(?urn) with shorthand properties
e.g. nie:title:0 nid3:contentType:24 nid3:contentType:43
2013-02-04 Martyn Russell <martyn@lanedo.com>
libtracker-data: Use g_hash_table_unref() not _destroy()
libtracker-data: Use a g_message() for cases where FTS is disabled
2013-02-04 Carlos Garnacho <carlos@lanedo.com>
tracker-needle: display snippet where available
Handle FTS queries more generically
FTS functions and other SQL functions like COUNT() don't
mix well, so handle it more generically by joining with the
FTS table at the outmost level, so those functions are handled
separately.
Use fts4 external content tables on Tracker FTS
FTS support on Tracker is now implemented using external content
support available in sqlite >= 3.7.9 FTS4.
FTS tables created this way fetch data from a single table/view,
for this purpose an intermediate view has been created to
interface with the FTS table. As this view is mainly queried by
ID (both when populating the FTS contents, and when querying
throught the FTS table), queries are fast enough on it.
As FTS indirectly points to the data on the real tables where tracker
data is stored, strings themselves are stored only once in the database,
so there is no impact in database size when compared to the previous
custom FTS code. Performance had little changes too from testing.
Detect at configure time whether FTS needs compiling
Recent sqlites (>3.7.9) with FTS support compiled have all
what Tracker requires to handle FTS, so only compile it
optionally if no FTS support was detected in the sqlite library.
Update FTS code
libtracker-data: Fix FTS initialization
Use a define that actually exists around tracker_fts_init(), which
also checked for the inverted value.
libtracker-data: Fix fts detection when constructing SQL
It would previously miss the FTS binding, depending on the order
of the clauses. so something like { ?u fts:match 'foo'; a nie:DataObject }
would return results whereas { ?u a nie:DataObject ; fts:match 'foo' }
didn't.
libtracker-fts: Fix reentrancy issues in the TrackerTokenizer
The internal TrackerParser must be per-cursor, as several cursors
may be opened simultaneously.
libtracker-data: Implement fts:snippet()
this function takes up to 3 optional parameters after the
object, the first 2 parameters are the starting/ending text
for the match (defaults to <b></b>), and the third one
modifies the ellipsis text (defaults to <b>...</b>)
libtracker-data: Perform FTS matching in subquery
The outer query will call the functions that require a full row
scan only with the elements returned by the inner query, so
it would perform better with OFFSET and LIMIT
fts: Respect max-words configuration setting
libtracker-data: Gather FTS changes to perform a single insert/update per ID
libtracker-data: remove useless FTS functions
commit/rollback should happen now with the rest of the
operations, and update_init() isn't necessary anymore.
libtracker-data: Cope with ontology changes in the FTS table
If a new property is with tracker:fulltextIndexed, we need to add
a new column for it in the FTS table.
libtracker-fts: Add a way to modify FTS table columns
As ALTER TABLE on virtual tables is only able to do renamings,
resort to creating a temporary table, and dump the old values
into the new table, which is then renamed.
libtracker-fts: Add tracker_offsets() sqlite function
This function takes the offsets() output and the FTS property
names in order to produce the output as expected by Tracker.
the fts:offsets() sparql function has been adapted to use it
underneath
libtracker-fts: Add the tracker_rank() sqlite function
This is now used in FTS searches to provide the rank based on
the matched columns and their respective property weight.
libtracker-data: Create several columns for FTS data
there is now one column per ontology property with FTS enabled,
the column name is that of the property.
libtracker-data: Add tracker_db_interface_sqlite_fts_delete_text()
This function removes from the FTS table the text from a resourceid/propid
pair.
libtracker-fts: Implement tokenizer FTS module using TrackerParser
libtracker-fts: Update to FTS4
The code isn't yet feature complete, but cleans up the tracker
integration with the FTS module, which means that fts3* files
are taken verbatim from sqlite, and are licenced as such.
2013-02-03 Jens Georg <mail@jensge.org>
tracker-extract: Port to new gupnp-dlna 2.0 API
GUPnP-DLNA 0.9/0.10 changed a lot internally and thus does not deliver a
GstDiscovererInfo anymore but instead has a new API that allows guessing the
profile from an existing GstDiscovererInfo, so most of the additional logic
can be dropped.
Fixes GB#691847, tracker-extract: Port to new gupnp-dlna API
2013-02-02 Martyn Russell <martyn@lanedo.com>
tracker-extract-playlist: Support nie:title for the playlist
Fixed GB#692409, New: playlist extractor should index playlist title if
available
2013-02-01 Martyn Russell <martyn@lanedo.com>
tracker-needle: Find documents (e.g. PDFs) when searching for creators/publishers
Fixes GB#692603, Search can not find pdf files using author names