dob.sk Blog

Just released passwd_exp version 1.2.11 with small feature fix for email seding. Download here.

passwd_exp is a highly customizable but easy to setup email password/account expiration notifier.

I’ve just published solr_pager a search component for Solr that should make paging easier. It’s main use should be by XSLT transformation (using XSLT writer), so one should not bother anymore with slow recursive XSLT hacks and only apply fast templates on paging list in search result.

Initial version 0.2.0 can be found on http://devel.dob.sk/solr_pager. Documentation is bundled

I’ve tested it against development version of Solr 1.4, but I think it should works also with currently stable Solr 1.3 (If not let me know, I’ll try to fix it).

How it works:

After install Solr Pager and configuring Solr to use Pager component for results (see README file), one should simply pass pager parameter with number of pager starts that should be generated (pager=10) to search query and list of pager start will be returned in response, smth. like this:

<response>
 <lst name="pager">
 <lst name="pages">  <!-- list of all pages -->
   <int name="2">25</int>
   <int name="3">50</int>
   <int name="4">75</int> <!-- this is actual page -->
   <int name="5">100</int>
   <int name="6">125</int>
   <int name="7">150</int>
   <int name="8">175</int>
   <int name="9">200</int>
   <int name="10">225</int>
   <int name="11">250</int>
 </lst>
 <int name="prev">50</int>  <!-- previous page, with start = 50 -->
 <int name="next">100</int>  <!-- next page, with start = 75 -->
 <int name="last">1225</int> <!-- last page, with start = 1225 -->
 <int name="actual">4</int> <!-- actual page number -->
 <int name="count">49</int> <!-- count of all pages -->
 </lst>
</response>

There is also another parameter pager.pre that sets how many proceeding pager starts should be generated, one pager start will be generated for actual page and all other pages will be pager starts of pages following the actual page.

Name is position of the page starting from 1 and node value is parameter (document position) that should be passed back to solr as value for start parameter.

It’s simple component, so it should have no bugs (HA HA), but if found any plz. report them to me. Also if you have some interesting and usable feature or patch that could be added to Solr Pager, again,don’t be ashamed and let me know.

I was dealing with problem how to remotely access subversion server in secure way. There are (as far i know) two basic ways to do this and I was not satisfied with neither of them, due to some security and hard-to-setup-or-maintain problems. So I have created my own way how to remotely access subversion repository in secure and easy way.

First, I’ll try to describe existing ways to do this and what problems they have. I’m not mentioning use of svnserve listening on widely open Internet port, because this is so insecure that I’m not going to talk about it.

Apache DAV Subversion module

Pros:

• You can use Apache modules  for user authentication (ie. via LDAP)
• Use PKI certificates to secure network communication channel
• Access rights managed by subversion configuration (svnserve.conf)

Cons:

• Apache ? What is it some dummy alternative to nginx or lighttpd ?
• You have to use Apache and its modules – this means one more layer with possible security problems and bugs (proper use PKI certificates will prevent most of this problems, but not all of them)
• iI you run single instance of Apache, it will have read-write permission to your subversion repositorories, so will all possibly buggy web apps running under Apache – thats huge security problem – one bug in your web CMS can give attacker full access to all your repositories – thats nothing I’m dreaming off
• Of course, you can run second instance of Apache to prevent previously mentioned problems, but this is second instance of Apache you have to take care of, configure it, maintain it and another point of possibly security problems.

So this points me to decision not to use Apache as server for SVN repositories (and also, I don’t want Apache on my system).

Subversion in tunnel mode (ie. ssh)

Pros:

• You have to setup almost nothing, works out of the box (the tunnel, repository access rights needs a little work)
• Secured by ssh

Cons:

• All repository users must have access rights to repository (ie. single group with group read-write access on repository files)
• Multiple repositories, means you should have separate group for each and manage access rights by adding/removing users from groups
• Must setup users umask and possibly run svnserve with setgid set to have consistent filesystem rights
• Again, each user needs read-write access to whole repository – this means each user can copy your whole repository, modify or even delete it – very secure
• Users must have system account (either local or ldap based, for ldap you have to carrefully limit which ldap users to import to system and what they can do (ie. only 10 from 1000 of employees work with svn)
• Each user should have shell access to system (but ok, i think it is posible to limit shell access only to use svnserve, but who does it ?)

Again nothing to think about, too many possible problems for me, I don’t trust my users and no one should! So I’ve created simple solution that I think solves most (maybe all) mentioned problems previous solutions have.

Subversion in tunnel mode via svnserve

It is important that this mode works exactly the same way as standard subversion tunnel mode works, there is nothing to hack on client side and client doesn’t even notices that it is not using standard tunnel mode – there is only one exception, subversion client will not cache users repository password. It’s a bit disturbing and uncomfortable but it gives you a bit more security, so right now I think it is another positive side effect of this solution.

How it works:

1. setup svnserve as usual (with all user accounts ans security permissions), under special user account (without shell access) on top of some repository, listening on some port on localhost (ie. 3690)
2. setup another account for ssh access with shell pointing to our tunnel script (see bellow).
3. for this account you can setup ssh key or certificate of anything your ssh allows you – this ssh can be distributed to svn users – I think you can event make it avaible on some internal site from where everyone can download it, because ssh is here used only for securing communication channel and not for authentication and authorization – this will do svnserve itself
4. and thats all. You can integrate subversion server with your LDAP Directory Server using eldap SASL extension, to make user management easier.
5. users can now access repository as usual using  svn+ssh://<server>/<directory> URL

Sample tunnel script:

#!/bin/bash -l
exec /bin/nc -q 1 -w 3 localhost 3690

For multiple repositories you should setup multiple subversion servers and have multiple access accounts, and modify tunnel script accordingly.

Pros:

• works out of the box and very easy to setup, easier than two previously mentioned usage scenarios)
• repository access fully managed by svnserve
• all things configured on place – svnserve.conf, here you can limit user access to certain repository tries and so on
• access secured by ssh (imho, very good piece of software)
• command line svn (and possibly also other clients) doesn’t caches users passwords

Cons:

• command line svn doesn’t caches users passwords
• maybe compared to HTTP access to subversion, problems on some LANs where only HTTP/HTTPS port is allowed to use. But this problem has also standard subversion tunnel mode and this can be bypassed by use of OpenVPN, which will be surely helpful also for other purposes.

Simple solution for not-so-trivial problem of copying xml node with all its attributes, but without child nodes.

Without the last condition, solution is easy, one would use <xsl:copy-of>, but with it things are a bit harder, you should do smt. like:

<xsl:copy select="."> <!-- this copies element name -->
   <xsl:copy-of select="@*"/> <!-- this copies all its attributes -->
   ...what..ever..you..want..here...
</xsl:copy>

And thats all, seems easy but it took me some time to realize it (And I must do it myself, because google didn’t helped me !).

Just released initial release of dmlquery tool, for doing DML queries on SQL database (or anything perl DBI abstraction layer can handle) via formated XML files.

Very useful for XML pipelining. With this and XML/XSLT transformations you can do very powerful thinks, like whole ETL tools for extracting data from one source (database, web…), transform it (using XSLT) and load to database (using dmlquery).

Hope it will help you.Click here for more http://devel.dob.sk/dmlquery/.

Note: Right now it has weak documentation, will try to improve it soon.