For a long time we've talked about stitching a relational database into Foswiki, mainly to speed up query searches. To keep things simple Foswiki uses a text database, which stores topics in lots of individual files. By default, searches are implemented by searching every one of these files individually for matching text. This isn't as bad as it sounds; searching is surprisingly fast. But it doesn't scale up terribly well, so people have coupled in various full-text search engines, such as Kinosearch, to accelerate text searches.

However there's been very little successful work yet on speeding up query searches – which is a shame, because query searches are heavily used in most wiki applications. The problem has always been that it's difficult to map from a Foswiki search query to any kind of structured search language, such as SQL.

One approach (championed by WikiRing member Sven Dowideit) has been to support a wider range of queries in the text search engine. Using the code I wrote to map query searches to regular expressions, Sven has been able to couple Foswiki to the MongoDB database, with some pretty impressive results. This is already available as the MongoDBPlugin. Other approaches have tried to map Foswiki queries to other database search models, such as XQuery, but with little success.

SQL is still the most common query language, and the RDBMS that support it are among some of the most mature and robust software on the planet. It's important that Foswiki has solutions that fall into that comfort zone. To that end I have implemented a new module, the DBIStoreContrib, that maps from the Foswiki query language to SQL. This allows us to use many of the existing RDBMS, such as MySQL, SQLite and PostgreSQL, to accelerate queries. By caching topics into the database in parallel to the text database, we avoid the risk of data loss that has made people nervous of other wiki solutions that use RDBMS, such as Mediawiki, while still maximising search performance.

It's early days for this module, and there's still lots to do to optimise how it works, so watch this space! If you are interested, and have a development environment, then you can check out the DBIStoreContrib from the Foswiki Subversion trunk.

Foswiki – the very best wiki application platform – is taking another giant leap forward….

Tags: database, foswiki, plugin, searching, sql, storage

What is legally permissible when adding a copyright notice to a derivative work?

In 2008 I stopped contributing to an open source project when it was pwned by a commercial interest. At that time a number of my original works existed in their source code repository, and still exist there pretty much as I left them 2 years ago, when I moved further development to a different repository. All of these works were released under the GPL, and carry my personal copyright notice, or that of a wider group of contributors who had worked on the project up to that date.

Since I left the project, the now-owners have redefined that contributors group, and have taken to retrospectively applying a different copyright to a number of these works. They have made some minor changes (mostly removing my personal branding, but also adding some scraps of new documentation) and added a dated copyright notice of their own that extends their claim back several years, despite the fact that (according to their own public records) no changes have been made until very recently.

Now, all the code and documentation is in their source code repository, so I would have no problem proving that I am the original copyright holder. However by making some very small changes, they have technically created a derivative work, and I don't have any problem with them adding a copyright notice to cover these new changes.

But what I find really objectionable is the dating of that new copyright notice back to a point well before the changes (5 years before, in at least one case).

This is obviously immoral, but the question is, is it actually illegal?

Update: it has come to light that in at least one case (not my code) the new copyright notice has been extended back to a date before the work was first published. I'm not sure whether to laugh or cry!

Tags: collaboration, community, copyright

The WebDAV support for Foswiki took another leap forward this morning, when we finished development of a new FilesysVirtualPlugin which supports different views for topics. This makes Foswiki topic content much more accessible to a wider range of tools.

Some time ago we (C-Dot Consultants and Kontextwork) jointly released the WebDAVContrib and FilesysVirtualPlugin, which together support access to Foswiki data via the WebDAV protocol. More recently we released the WebDAVLinkPlugin, which enhances the integration of WebDAV documents into Foswiki, by allowing links to WebDAV documents to be edited using native applications on the client, such as the Microsoft Office™ suite.

That's great for working with attachments, but what about topics? Up to now, if you wanted to edit a topic, you had to edit the raw Topic Markup Language (TML) that the topic was stored in. Using Microsoft Word to edit TML is like using the space shuttle to commute 2km to work. Life would be much simpler if we could feed Word with a format it understands, like HTML.

We've been able to WYSIWYG edit Foswiki topics in the browser for a long time now; so the technology to convert topic content to other formats and back again exists; all we needed was a way to present this through WebDAV.

What we need to do is to give the WebDAV user a way to select what format they want the topic in. We've done this by allowing the same topic content to be mapped to more than one file entry in a WebDAV directory listing; each file entry is referred to as a "view" (because it's just a different way of looking at the same data).

The FilesysVirtualPlugin is configured with a list of views that it supports; for example, txt, html, json. When it is asked for a list of files in a Foswiki web, it generates a file entry for each of the different views, so the topic MyTopic ends up with the entries:

• MyTopic.txt
• MyTopic.html
• MyTopic.json

These file entries can be read and written just like normal files; the FilesysVirtualPlugin takes care of mapping the content back to TML. Because these file entries are all views of the same data, then if you edit MyTopic.txt, then MyTopic.html automatically changes too, and vice-versa.

Besides txt, html, and raw, two meta-data views have been provided, json and perl. These views allow you access to topic meta-data. It's easy to add new views, and we foresee other useful view types, such as xml and yaml coming along later.

WebDAVContrib, FilesysVirtualPlugin and the companion WebDAVLinkPlugin are available from Kontextwork.

Tags: ajax, foswiki, webdav, wysiwyg

For a long time we've talked about stitching a relational database into Foswiki, mainly to speed up query searches. To keep things simple Foswiki uses a text database, which stores topics in lots of individual files. By default, searches are implemented by searching every one of these files individually for matching text. This isn't as bad as it sounds; searching is surprisingly fast. But it doesn't scale up terribly well, so people have coupled in various full-text search engines, such as Kinosearch, to accelerate text searches.

Tags: database, foswiki, plugin, searching, sql, storage

What is legally permissible when adding a copyright notice to a derivative work?

Tags: collaboration, community, copyright

The WebDAV support for Foswiki took another leap forward this morning, when we finished development of a new FilesysVirtualPlugin which supports different views for topics. This makes Foswiki topic content much more accessible to a wider range of tools.

Tags: ajax, foswiki, webdav, wysiwyg

Ohloh Charts

… show that we are doing quite fine actually. Note also, that the number of active contributors is higher than ever before, even on the old project.

(red: foswiki, green: twiki)

Thanks

I think im gonna start a wiki for web hosting. Could any one point me at a link for a how to on getting started. Which interface to use ..etc.

Thank you

Paula J

Some other sources underwriting your results

Sven Dowideit made a reasonable comment on IRC as well, with links to some objective sources that confirm your analysis:

11:31 <@SvenDowideit> http://foswiki.org/Tasks/HallOfFame and http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/HallOfFame shows those stats reasonably too

Quite dramatic indeed!

Ohloh Charts

… show that we are doing quite fine actually. Note also, that the number of active contributors is higher than ever before, even on the old project.

(red: foswiki, green: twiki)

For a long time we've talked about stitching a relational database into Foswiki, mainly to speed up query searches. To keep things simple Foswiki uses a text database, which stores topics in lots of individual files. By default, searches are implemented by searching every one of these files individually for matching text. This isn't as bad as it sounds; searching is surprisingly fast. But it doesn't scale up terribly well, so people have coupled in various full-text search engines, such as Kinosearch, to accelerate text searches.

However there's been very little successful work yet on speeding up query searches – which is a shame, because query searches are heavily used in most wiki applications. The problem has always been that it's difficult to map from a Foswiki search query to any kind of structured search language, such as SQL.

One approach (championed by WikiRing member Sven Dowideit) has been to support a wider range of queries in the text search engine. Using the code I wrote to map query searches to regular expressions, Sven has been able to couple Foswiki to the MongoDB database, with some pretty impressive results. This is already available as the MongoDBPlugin. Other approaches have tried to map Foswiki queries to other database search models, such as XQuery, but with little success.

SQL is still the most common query language, and the RDBMS that support it are among some of the most mature and robust software on the planet. It's important that Foswiki has solutions that fall into that comfort zone. To that end I have implemented a new module, the DBIStoreContrib, that maps from the Foswiki query language to SQL. This allows us to use many of the existing RDBMS, such as MySQL, SQLite and PostgreSQL, to accelerate queries. By caching topics into the database in parallel to the text database, we avoid the risk of data loss that has made people nervous of other wiki solutions that use RDBMS, such as Mediawiki, while still maximising search performance.

It's early days for this module, and there's still lots to do to optimise how it works, so watch this space! If you are interested, and have a development environment, then you can check out the DBIStoreContrib from the Foswiki Subversion trunk.

Foswiki – the very best wiki application platform – is taking another giant leap forward….

Tags: database, foswiki, plugin, searching, sql, storage

What is legally permissible when adding a copyright notice to a derivative work?

In 2008 I stopped contributing to an open source project when it was pwned by a commercial interest. At that time a number of my original works existed in their source code repository, and still exist there pretty much as I left them 2 years ago, when I moved further development to a different repository. All of these works were released under the GPL, and carry my personal copyright notice, or that of a wider group of contributors who had worked on the project up to that date.

Since I left the project, the now-owners have redefined that contributors group, and have taken to retrospectively applying a different copyright to a number of these works. They have made some minor changes (mostly removing my personal branding, but also adding some scraps of new documentation) and added a dated copyright notice of their own that extends their claim back several years, despite the fact that (according to their own public records) no changes have been made until very recently.

Now, all the code and documentation is in their source code repository, so I would have no problem proving that I am the original copyright holder. However by making some very small changes, they have technically created a derivative work, and I don't have any problem with them adding a copyright notice to cover these new changes.

But what I find really objectionable is the dating of that new copyright notice back to a point well before the changes (5 years before, in at least one case).

This is obviously immoral, but the question is, is it actually illegal?

Update: it has come to light that in at least one case (not my code) the new copyright notice has been extended back to a date before the work was first published. I'm not sure whether to laugh or cry!

Tags: collaboration, community, copyright