Archive

Archive for the ‘devel’ Category

perl-XML::LibXML + global external entity loader

May 15th, 2011 No comments

Just created a quick patch against perl XML::LibXML module, that adds global external entity loader support. Till now it was only possible to have per instance entity loader, but this is not enough if you want i.e., XML::LibXSLT to also use yours entity loader for imports, and input callbacks doesn’t suit all your needs.

Usage is simple:

XML::LibXML::externalEntityLoader(\&_entity_handler);

where _entity_handler is subroutine like by option ext_ent_handler described in (http://search.cpan.org/dist/XML-LibXML/lib/XML/LibXML/Parser.pod#PARSER_OPTIONS).

Note: when you define global entity loader, per instance entity loader is simply ignored.

You can download this patch  from http://devel.dob.sk/patches/perl-XML::LibXML+global_entity_loader-0.1.diff. Just download XML::LibXML from cpan, patch it and install. I’ll try to push it to CPAN if possible ;-)

Categories: devel, perl, xml Tags: ,

Oracle VARNUM/NUMBER encoding in C

April 28th, 2011 No comments

Just a short example C implementation of Oracle VARNUM/NUMBER type encoding, in case you could not use library functions. I didn’t found it anywhere when needed (and afterwards I found it doesn’t helps me at all due to little ‘problem’ in ora :( ) and Oracle documentation is pretty unclear about how-to do it.

 

typedef	struct
{
	uint8_t len;
	uint8_t exp;
	uint8_t man[20];
} VARNUM_t;
static void setVARNUM(VARNUM_t *varnum, int64_t value)
{
	int8_t digits = 0;
	uint64_t u_value = 0;
	/* init */
	memset(varnum, 0, sizeof(*varnum));
	varnum->len	= 1;	//start len
	/* check sign */
	if(value >= 0)
	{
		u_value		= value;
		varnum->exp	= trunc(log(u_value) / log(100)) + 128 + 65;
	}
	else
	{
		u_value		= value * - 1;
		varnum->exp	= trunc(log(u_value) / log(100)) + 128 + 65;
		varnum->exp	= ~varnum->exp;
	}
	/* count value digits */
	digits = trunc(log(u_value) / log(10));
	digits = ((digits / 2) + 1);
	/* mantisa */
	for(; digits >= 0 && varnum->len <= 20; digits--)
        {
                uint64_t v_tmp = 0;
                /* prevent INT overflow for too much digits */
                if(digits > 0)
		{
			uint64_t v_pow = powl(100, digits - 1);
			v_tmp = ((u_value / v_pow) / 100) % 100;
		}
		else
			v_tmp = u_value % 100;
		if(!v_tmp)
			continue;
		/* +1 for positive, subtract 101 for negative */
		v_tmp = (value < 0 ? (101 - v_tmp) : (v_tmp + 1));
                varnum->man[varnum->len - 1] = v_tmp;
		varnum->len++;
	}
	/* terminator byte for negative value */
	if(value < 0 && varnum->len <= 20) 	{
                varnum->man[varnum->len - 1] = 102;
		varnum->len++;
	}
	return;
}

NUMBER is just the same as VARNUM but without len member, thus is one byte shorter.

Note: this is just a quick implementation that works, there are some easy optimizations possible…

Categories: devel, rdbms Tags: ,

perl utf8 and using Digest functions

April 27th, 2011 No comments

I’ve implemented new neat feature (to store unique content only once in cache) to my perl based etl tool and suddenly it started to print sometimes ‘Wide character in subroutine entry perl warning in sha1_hex call. As if this was not enough processed content after being stored in cache  started to be utf-8 corrupted in comparition to the one stored in the cache.

It took few funny hours with of playing with perl, till I’ve found that sha1_hex function somehow destroyed parameter content but in such a beautiful way, that it was almost impossible to detect it. What the best is the cached content was output of LibXML toString() function, but the XML tree itself (or what) has been also corrupted. Well it must one of that  ‘perl secrets’.

Afterwards Google given me some explanation of this activity – I’ve found similar problem reported for md5_sum function from digest::md5 package.

So finally to fix that problem, one must call encode_utf8() on sha1_hex parameter to let sha1_hex work on this copied content.

 

Categories: devel Tags: ,

Solr Pager 0.2.2 released

December 20th, 2009 No comments

Released solr_pager 0.2.2. This release fixes little bug that prevented solr_pager to work in some configurations (ie. with standard search handler). Well, Solr documentation did not mentioned that it can pass empty (null) result set to my component.

Also added some sanity checks to prevent similar situation again.

New version can be found on http://devel.dob.sk/solr_pager, enjoy.

Categories: devel, pager, solr, xml, xsl Tags:

Solr Pager 0.2.1 released

December 9th, 2009 No comments

Released solr_pager 0.2.1, an feature update with improved first/last page navigation. Now they appear in result only when first/last page is not visible in generated paging list.

This version can be found on http://devel.dob.sk/solr_pager, for more informations about how it works see this post.

Categories: devel, pager, solr, xml, xsl Tags: