Tardate 2016

tree-sync.pl has had a little update thanks to a contribution from Dave Stafford.

• Added option 'diff' to view only changes


• Added option 'ignore' to exclude certain extensions


• Added option 'brief' to remove src/dest directories in report view to make cleaner output

In the 2.4 release I've also added the 'force' option to control read-only file handling.

tree-sync.pl is a bit of a geeky tool, suitable for cases where you want detailed control over a file-based copy/mirror/backup process from the command-line. There are many flashier solutions available these days, but if tree-sync scratches your itch - go for it!

tree-sync.pl continues to be available on CPAN, but from the 2.4 release I have moved the source to github. If you have changes to suggest or bugs to fix, please go ahead and join this project.

Unless you get to work with regular expressions everyday, the details can easily blur. You remember enough to know when a regex could be useful, but not enough to write it without reaching for a reference manual.

There are a few tools out there to help you write and understand regular expressions, including some IDEs that can provide assistance (like Komodo).

Sometimes you just want a quick explanation of a regular expression you might have seen in some code. The YAPE::Regex::Explain perl module is one tool that helps you do just that. Turning the module into a simple command-line tool is a simple one-liner:

#!/usr/bin/perl -w
print YAPE::Regex::Explain->new($ARGV[0])->explain;

I've also turned it into a simple CGI utility for those times when my regex memory fails me. You can use it here: regexplainr (sorry, you may find that site offline but here's the source code).

In a previous post, On Parsing CSV and other Delimited/Quoted Formats, I used the following regular expression to parse a whitespace-delimited string:

"([^"]+?)"\s?|([^\s]+)\s?|\s

Regexplainr produces the following commentary:

The regular expression:

(?-imsx:"([^"]+?)"\s?|([^\s]+)\s?|\s)

matches as follows:

NODE EXPLANATION
----------------------------------------------------------------------
(?-imsx: group, but do not capture (case-sensitive)
(with ^ and $ matching normally) (with . not
matching \n) (matching whitespace and #
normally):
----------------------------------------------------------------------
" '"'

----------------------------------------------------------------------
( group and capture to 1円:
----------------------------------------------------------------------
[^"]+? any character except: '"' (1 or more
times (matching the least amount
possible))
----------------------------------------------------------------------
) end of 1円
----------------------------------------------------------------------
" '"'
----------------------------------------------------------------------
\s? whitespace (\n, \r, \t, \f, and " ")
(optional (matching the most amount
possible))
----------------------------------------------------------------------
| OR
----------------------------------------------------------------------
( group and capture to 2円:
----------------------------------------------------------------------
[^\s]+ any character except: whitespace (\n,
\r, \t, \f, and " ") (1 or more times
(matching the most amount possible))
----------------------------------------------------------------------
) end of 2円
----------------------------------------------------------------------
\s? whitespace (\n, \r, \t, \f, and " ")
(optional (matching the most amount
possible))
----------------------------------------------------------------------
| OR
----------------------------------------------------------------------
\s whitespace (\n, \r, \t, \f, and " ")
----------------------------------------------------------------------
) end of grouping
----------------------------------------------------------------------

(Permalink to this regexplanation)
[フレーム]
Book tip: (thanks to Tony) O'Reilly's Mastering Regular Expressions. Available on google books, and also from Amazon.

Seasons greetings to all! Of course, a few days off means time for some of those special projects that have been waiting in the wings;-)

I run a cvs repository for my personal use at home (migrating to subversion is a project for another xmas break). There are two stupid mistakes in the setup I've wanted to fix for a long time.

1. It is running on the main machine I use for testing and development, which can lead to some tricky catch-22 situations. Duh!
2. I've always used the servername to access the repository, which makes relocation a pain. This seems to be a problem common to the many development shops I have seen. Gold stars for actually using source control, but brickbats for not thinking of using a virtual host name before finding yourselves tied to a specific host!

The Relocation

Actually, by moving to a new server I mean old laptop .. it is easy to write-off a 256Mb 600MHz machine these days, but they really do make decent "network utility" servers! Running Linux of course. Ideal for just sitting in the corner and running your crucial network services day in, day out. Like cvs.

Since I was migrating the cvs repository from another Linux machine, the move was a piece of cake. The following notes assume Red Hat (or Oracle Linux), which is what I am running. It also assumes the cvs repository is at /cvsroot and the cvs user account is called cvs (just substitute your actual details where approriate).

CVS Setup - New Server
Most Linux distributions will already have the cvs server installed (but not configured or running). Since I'm on Red Hat, I can check for the cvs package using rpm:

$ rpm -qa | grep cvs
cvs-1.11.2-27

If you can't find the server package, download and install as appropirate for your distro.

Setting up the CVS Service
Create a new cvs user and group to own the repository.

[root@new server]# /usr/sbin/groupadd cvs
[root@new server]# /usr/sbin/useradd -g cvs cvs
[root@new server]# passwd cvs

Check that the service name is listed in /etc/services

$ grep cvspserver /etc/services
cvspserver 2401/tcp #cvs pserver process

To get the cvs server under xinetd control, create /etc/xinetd.d/cvspserver:

service cvspserver
{
socket_type = stream
protocol = tcp
wait = no
user = root
passenv = PATH
server = /usr/bin/cvs
server_args = --allow-root=/cvsroot -f pserver
}

Move the Repository Files
Normally at this point you would initialise a new repository if you are setting up a server from scratch (cvs -d /cvsroot init). However in this case I'm moving the repository files from the old to the new server. For me, this meant tar/gzip of the /cvsroot, ftp the archive to the new machine, and unzip to re-create /cvsroot.

[root@old server]# tar -zcvf cvsroot.tar.gz /cvsroot
[root@new server]# cd /
[root@new server]# ftp {file from old server}
[root@new server]# tar -zxvf cvsroot.tar.gz

Starting the Server

[root@new server]# /etc/init.d/xinetd restart

Voila!

Switching to a Virtual Hostname

OK, now we get to the main point of the post - using cvs with a "virtual hostname". I mean of course using a name that belongs to the cvs service rather than any particular server, which will make future relocations to new servers a breeze if it becomes necessary.

So where I used to access "server1.mydomain.com", I will now use "cvs.mydomain.com". These names can be resolved by DNS or hosts file. They could actually map to different IPs (i.e. use virtual IP address also), but in my case I have just setup "cvs.mydomain.com" as an alias for my new server's IP address.

To preconfigure the cvs environment for Linux users I add the following to /etc/profile:

export EDITOR=/bin/vi
export CVSROOT=:pserver:$USER@cvs.mydomain.com:/cvsroot

Migrating CVS Clients
Now what about all the cvs "clients"? Strict protocol would suggest you should make sure everything is checked in before the server move, and check out a fresh copy from the new server when it is ready. This is not really a nice way to go for at least two reasons:

1. You'll probably miss something that needs checking in
2. If you have directories with a mix of cvs controlled/non-controlled files you have an interesting mix-and-match problem to get everything back in sync

Fortunately there are some tricks you can employ to do an "in-place" migration of modules checked out on the client. Basically we just need to update the CVS/Root files with the details of the new server.

Here's a quick perl recipe for updating all of the CVS/Root files under the current directory in Linux:

perl -i.orig -pe "s/myacct\@server1.mydomain.com/mynewacct\@cvs.mydomain.com/g;" $(find . -name Root)

On Windows, you can use the extended FOR syntax to descend the directory tree:

for /R %r in (Root?) do perl -i.orig -pe "s/myacct\@server1.mydomain.com/mynewacct\@cvs.mydomain.com/g;" %r

These perl one-liners (-e) loop line-by-line (-p) through the file, modifying it in-place while also creating a backup file with the .orig suffix (-i.orig). Each line of the file is evaluated with the global search and replace regular expression (s///g).

About time to post about something ... too many OpEd pieces of late!

An interesting question came up on the soaplite mailing list concerning how to modify the namespaces registered in the SOAP envelope. Documentation is not especially clear on this point.

Of course, a quick hack is to attach a full namespace to an element directly, as in:
SOAP::Data->name('itemName')->attr({'xmlns:mns' => 'http://my.namespace'})

With version SOAP::Lite 0.65 and above, the register_ns serializer method helps to correctly construct the envelope, as shown in the following example:

#!/usr/bin/perl -w
# $Id$
#

use strict;

#SOAP::serializer->register_ns requires 0.65
use SOAP::Lite 0.65 +trace => 'debug';

my $soap = SOAP::Lite
->proxy( 'http://localhost/blah/DummyService' );

my $serializer = $soap->serializer();
$serializer->register_ns( 'http://my.namespace', 'mns' );

my $som = $soap->call(
SOAP::Data
->name('mns:test')
=> SOAP::Data->name('mns:description' => 'an item in my namespace')->type('mns:mytype')
);

This generates the following SOAP request:

<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:mns="http://my.namespace"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<mns:test>
<mns:description xsi:type="mns:mytype">an item in my namespace</mns:description>
</mns:test>
</soap:Body>
</soap:Envelope>

I mentioned in Log Buffer #54 that "Patrick Wolf stumbled across an 11g feature that means DBA's may put Java/SOA guys out of work".

Well I finally got myself setup with an 11g test system and the Native Web Services was the first thing I jumped at testing.

Conclusions first?

• Although perhaps not a blockbuster new feature, Native Web Services provide an easy solution for exposing data services into a SOAP-oriented application stack.


• For 11.1.0.5 (only), see Metalink Note 444191.1 for updated installation instructions.


• The auto-generated XMLSchema and WSDL have some goofy features (like hyphens in method names) that may break SOAP client toolkit smarts.


• The is no real control over the generated XML payload, which will possibly limit the usefulness of Native Web Services for Enterprise SOA initiatives.


• The security model is simple but effective. The underlying database user role model provides authentication and access control.


• The ability to call arbitrary packages/procedures/functions is perhaps the most powerful feature. No incremental coding or configuration is required for each method. A good service interface is thus just a matter of careful procedure design and construction.


• I am less convinced that the ability to execute arbitrary SQL via Native Web Services is a good thing. It's not SOA, and it's not a good replacement for JDBC, ODP etc. Seems like just an invitation for bad hacks and shortcuts..

So, after a couple of hours of playing, I think maybe the Java/SOA guys don't have to panic just yet.. ;)

Here's the blow-by-blow, where I describe the setup gotchas I hit, and my simple SOAP::Lite testing with Perl ..

Initial Scan..

It took me a few moments to orient myself in the documentation; rather than being a distinct major new feature, Native Web Services are an enhancement of XML DB, and so the requisite documentation is mainly found in the XML DB Developer's Guide.

The architecture of Native Web Services is quite simple. The main moving part is a servlet called 'orawsv' which brokers SOAP 1.1 requests and handles automatic WSDL generation. Out of the box, it is not enabled.

The feature works in two main ways:

1. An arbitrary SQL DML request can be sent to the main 'oawsv' endpoint, and the results are returned as XML in the SOAP response.


2. Procedures and functions may be invoked directly over SOAP.

First Up - Configuration

Configuring Native Web Services is simply a matter of enabling the orawsv servlet, and then granting user access through role assignment.

This is covered in Chapter 33 of the docs.

I first tried this with 11.1.0.5, and found that the role names mentionedd in the docs do not match (XDB_WEBSERVICES instead of XDBWEBSERVICES and so on). For an 11.1.0.5 install, refer to Metalink Note 444191.1 for corrected installation instructions. In 11.1.0.6, role names correctly match the docs.

The second thing I discovered (actually, a problem I created for myself then had to fix!) is that the servlet name does matter! The SERVLET_NAME (as in the setup code below) must be 'orawsv'.

Thirdly, although you can set the URL pattern to anything you wish (and calls work), the auto-generated WSDL always assumes 'orawsv' when generating endpoint addresses. Effectively this means you must use 'orawsv' as the URL pattern also.
NB: thanks to Christopher Burke for alerting me to the WSDL situation.

DECLARE
SERVLET_NAME VARCHAR2(32) := 'orawsv';
BEGIN
DBMS_XDB.deleteServletMapping(SERVLET_NAME);
DBMS_XDB.deleteServlet(SERVLET_NAME);
DBMS_XDB.addServlet(NAME => SERVLET_NAME,
LANGUAGE => 'C',
DISPNAME => 'Oracle Query Web Service',
DESCRIPT => 'Servlet for issuing queries as a Web Service',
SCHEMA => 'XDB');
DBMS_XDB.addServletSecRole(SERVNAME => SERVLET_NAME,
ROLENAME => 'XDB_WEBSERVICES',
ROLELINK => 'XDB_WEBSERVICES');
DBMS_XDB.addServletMapping(PATTERN => '/orawsv/*',
NAME => SERVLET_NAME);
END;
/
GRANT XDB_WEBSERVICES TO SCOTT;
GRANT XDB_WEBSERVICES_OVER_HTTP TO SCOTT;
GRANT XDB_WEBSERVICES_WITH_PUBLIC TO SCOTT;

I couldn't quite figure out exactly why the 'orawsv' SERVLET_NAME is significant. It apparently just is, providing an explicit reference to the required servlet. The DBMS_XDB.addServlet docs are unfortunately terse and unenlightening.

So that all took me an hour more than it should. But not to worry - I am now up and running!

Now What Can It Do? Testing a Function Call by Web Services..

I was particularly interested to see a procedural interface exposed via Web Services, so my first little test was to call a database function from Perl (using SOAP::Lite). I created this very simple function in SCOTT:

CREATE OR REPLACE FUNCTION empcount
RETURN NUMBER IS
emp_count number;
BEGIN
SELECT count(*) INTO emp_count FROM emp;
RETURN emp_count;
END;
/

This becomes available at the endpoint 'http://server:port/orawsv/SCOTT/EMPCOUNT', and the auto-generated WSDL is available at 'http://server:port/orawsv/SCOTT/EMPCOUNT?wsdl'.

Next I knocked up a bare-bones Perl client. Note the redefined get_basic_credentials method to provide the basic authentication credentials that are required.

#!/usr/bin/perl -w
#
use SOAP::Lite +trace => 'debug';
sub SOAP::Transport::HTTP::Client::get_basic_credentials {
return 'scott' => 'tiger';
}
my $serviceNs = 'http://xmlns.oracle.com/orawsv/SCOTT/EMPCOUNT';
my $soap = SOAP::Lite
->proxy('http://localhost:8080/orawsv/SCOTT/EMPCOUNT');
my $som = $soap->call( SOAP::Data->name('SNUMBER-EMPCOUNTInput')->attr({'xmlns' => $serviceNs}) );
print "The response from the server was:" . $som->result . "\n";

And the answer is .. 14;) [see the end notes below for the request/reply transcript].

As you can see, pretty simple, but it should be simpler. One liners should be possible, like this:

print SOAP::Lite-> service('http://scott:tiger@localhost:8080/orawsv/SCOTT/EMPCOUNT?wsdl')->EMPCOUNT();

... but you'll notice that the 11g generated method names (like 'SNUMBER-EMPCOUNTInput') have annoying hyphens, which subverts the smart typing and dynamic binding that kits like SOAP::Lite are capable of. Doh!

Closest I can get is this:

print "The response from the server was: ";
print SOAP::Lite
->uri('http://xmlns.oracle.com/orawsv/SCOTT/EMPCOUNT')
->proxy('http://localhost:8080/orawsv/SCOTT/EMPCOUNT')
->call ( 'SNUMBER-EMPCOUNTInput' )
->result;

Endnotes - EMPCOUNT SOAP Request and Response

{The request ... }
SOAP::Transport::HTTP::Client::send_receive: POST http://localhost:8080/orawsv/SCOTT/EMPCOUNT HTTP/1.1
Accept: text/xml
Accept: multipart/*
Accept: application/soap
Content-Length: 461
Content-Type: text/xml; charset=utf-8
SOAPAction: "http://xmlns.oracle.com/orawsv/SCOTT/EMPCOUNT#SNUMBER-EMPCOUNTInput"

<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<SNUMBER-EMPCOUNTInput xmlns="http://xmlns.oracle.com/orawsv/SCOTT/EMPCOUNT" xsi:nil="true" />
</soap:Body>
</soap:Envelope>

{The response ... }
SOAP::Transport::HTTP::Client::send_receive: HTTP/1.1 200 OK
Server: Oracle XML DB/Oracle Database
Content-Type: text/xml; charset=UTF-8
Client-Date: 2007年8月13日 16:06:28 GMT
Client-Peer: 127.0.0.1:8080
Client-Response-Num: 1
Client-Transfer-Encoding: chunked
DAV: 1,2,<http://www.oracle.com/xdb/webdav/props>
MS-Author-Via: DAV

<?xml version="1.0" ?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
<EMPCOUNTOutput xmlns="http://xmlns.oracle.com/orawsv/SCOTT/EMPCOUNT">
<RETURN>14</RETURN>
</EMPCOUNTOutput>
</soap:Body>
</soap:Envelope>

Take part in the 2007 Perl Survey! The Perl Survey is an attempt to capture a picture of the Perl community in all its diversity. No matter what sort of Perl programmer you are, they'd love to hear from you.

The survey can be found at: http://perlsurvey.org/

It only takes about 5 minutes to complete, and will be open until September 30th, 2007.

Security Now! #101 with Steve Gibson and Leo Laporte covered the deceptively simple challenge of differentiating human from non-human automated clients (great podcast as always .. check it out). Commonly this is done with distorted text like the sample on the right.

The Official CAPTCHA Site has a wealth of information about this field, including discussion of the relay attack that has the potential to defeat any "are you a human?" test (because it enrolls unwitting human accomplices to do the work).

The coolest piece of work though is the reCAPTCHA project. This is a project of the School of Computer Science at Carnegie Mellon University, and it provides a public service for plugging-in a CAPTCHA to your site. But unlike other systems that are just wasting 10 seconds of your time, this system is actually digitizing public domain archives at the same time (getting you to fix the translation of words that have defied the best OCR software)!

Perl is one language that you can easily use reCAPTCHA with, using Andy Armstrong's Captcha::reCAPTCHA module.

After installing the module, it just takes a few minutes to register your site and setup a test page. I built a quick reCAPTCHA cgi in perl .. you can try it out here (sorry, currently may find it offline but get the script source here: myReCaptcha.pl. I think I spent 10 minutes setting this up, and then an hour playing with it;) All in the name of digitizing historical works...

But I'm afraid the humour archive has the best captcha ever ..

Antony Reynolds' recent Diving for Perls with WSIF post gave a great example of how you can use HTTP bindings to call perl CGI scripts from Oracle BPEL Process Manager.

If your perl code is not already available to be called in this way, then what to do? Certainly the "ideal" would be make it available as a native Web Service and do away with any special binding. Thanks to the SOAP::Lite module, this is actually quite easy to do.

I'm going to walk through an example of how to take some aribitrary perl code, wrap it as a Web Service, and then call it from a BPEL process. See the diagram:

The Perl Code

In this example, there's really only one bit of code that "matters" ... a helloWorld function. I'm going to start with this wrapped in a perl class module called HelloWorld.pm. As you'll see shortly, wrapping the business functionality in a class is a good idea because it allows automatic dispatching from the Web Services interface.

$ cat HelloWorld.pm
#!/usr/bin/perl -w
use strict;
package HelloWorld;
our (@ISA, @EXPORT, $VERSION);
use Exporter;
$VERSION = 1.00;
@ISA = qw(Exporter);
@EXPORT = qw( helloWorld );

sub helloWorld {
my ($self,$foo) = @_;
return 'Hello ' . $foo;
}
1;

Important to note that while the code here contains some of the module niceties, it doesn't make any reference to SOAP, CGI or BPEL. It's plain perl. We can prove that with a little perl test program:

$ cat helloWorld.pl
#!/usr/bin/perl -w
use strict;
use HelloWorld;
print HelloWorld->helloWorld( 'Sunshine' );

$ perl helloWorld.pl
Hello Sunshine
$

The SOAP Interface

The dynamic typing of perl and flexibility of the SOAP::Lite module really live up to the make simple things easy motto. In three lines of code we have a SOAP CGI server for our HelloWorld class (that's why I made it a class;)

$ cat HelloWorld.cgi
#!/usr/bin/perl -w
use HelloWorld;
use SOAP::Transport::HTTP;
SOAP::Transport::HTTP::CGI
->dispatch_to('HelloWorld')
->handle;

That was so easy, there must be a catch right? Well yes, one comes to mind: the reply message elements will necessarily have some generated names (like "s-gensym3") since there is nothing in our code to provide any guidance for things like the "name" of function return value elements.

Testing SOAP Client-Server

After dropping HelloWorld.cgi and HelloWorld.pm into my apache cgi-bin, I'm ready to test the SOAP service over HTTP. We can whip up a client in no time:

$ cat HelloWorldWSClient.pl
#!/usr/bin/perl –w
use SOAP::Lite;

my $soap = SOAP::Lite
->readable(1)
->uri('urn:HelloWorld')
->proxy('http://localhost:8000/cgi-bin/HelloWorld.cgi');

my $som = $soap->helloWorld(
SOAP::Data->name('name' => 'Sunshine')
);
print "The response from the server was:\n".$som->result."\n";

$ perl HelloWorldWSClient.pl
The response from the server was:
Hello Sunshine
$

If we sniff the network or route this request via a tool like org.apache.axis.utils.tcpmon, we can see the outbound request and incoming reply:

Creating a WSDL file

Alas, perl's flexibility means that automatically generating a WSDL for our SOAP service is easier said than done. Unlike in strongly-typed languages, perl methods can take an arbitrary number of parameters of arbitrary type ... whereas of course a Web Service should have a very clearly defined interface.

I think one of the best approaches at present for generating WSDL in perl is the Pod::WSDL module. I'll perhaps leave that for another blog entry. For now lets just assume we'll manually create a WSDL for our service:

$ cat HelloWorld.wsdl
<?xml version="1.0" encoding="UTF-8"?>
<wsdl:definitions targetNamespace="http://localhost:8000/HelloWorld" xmlns:impl="http://localhost:8000/HelloWorld" xmlns:wsdlsoap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:tns1="http://localhost:8000/HelloWorld">

<wsdl:message name="helloWorldRequest">
<wsdl:part name="name" type="xsd:string" />
</wsdl:message>

<wsdl:message name="helloWorldResponse">
<wsdl:part name="s-gensym3" type="xsd:string" />
</wsdl:message>

<wsdl:portType name="HelloWorldHandler">
<wsdl:operation name="helloWorld" parameterOrder="name">
<wsdl:input message="impl:helloWorldRequest" name="helloWorldRequest" />
<wsdl:output message="impl:helloWorldResponse" name="helloWorldResponse" />
</wsdl:operation>

</wsdl:portType>

<wsdl:binding name="HelloWorldSoapBinding" type="impl:HelloWorldHandler">
<wsdlsoap:binding style="rpc" transport="http://schemas.xmlsoap.org/soap/http" />

<wsdl:operation name="helloWorld">
<wsdlsoap:operation soapAction="" />
<wsdl:input name="helloWorldRequest">
<wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://localhost:8000/HelloWorld" use="encoded" />
</wsdl:input>
<wsdl:output name="helloWorldResponse">
<wsdlsoap:body encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" namespace="http://localhost:8000/HelloWorld" use="encoded" />
</wsdl:output>
</wsdl:operation>

</wsdl:binding>

<wsdl:service name="HelloWorldHandlerService">
<wsdl:port binding="impl:HelloWorldSoapBinding" name="HelloWorld">
<wsdlsoap:address location="http://localhost:8000/cgi-bin/HelloWorld.cgi" />
</wsdl:port>
</wsdl:service>

</wsdl:definitions>

Invocation from a BPEL Process

Now you have all the bits in place to invoke your Perl code as a fully-fledged Web Service from within BPEL. I won't go into this in detail here because it is the standard Web Service invocation process. Just add an "invoke" activity in your process and point it to a partner link defined based on the WSDL generated above.

Once you have deployed your process, you can test it from the BPEL Console. Here's an example of the invoke activity in one of my tests:

Conclusion?

Hopefully I've shown that exposing perl code as a Web Service is actually pretty simple. Once done, the code is then available for use by standards-based tools like Oracle BPEL Process Manager.

There are a couple of consideration to bear in mind though:

1. SOAP::Lite provides some great hooks for automatically generating a SOAP interface, however these come with the caveat that reply message elements will necessarily have some "generated" names
   
2. Automatic WSDL generation is confounded by perl's dynamic typing. Modules like Pod::WSDL provide some good solutions though.
   

In my last post I said one of my "Top 10" wishes for OTN was to be able to get an RSS feed of posts by a specified member to the Oracle Forums.

At first it may sound a bit narcissistic to have a feed that allows you to follow what you have written yourself!

It was my exploration of Jaiku that prompted the thought. Since "presence" is their big thing, I've been experimenting to see what's its like to have Jaiku aggregate all your web activity. So far it looks really cool - I love the interface. Must say that I'm not sure how useful Jaiku may turn out to be in the long run ... I suspect it works best if you have a whole lot of your friends also using it. NB: the Jaiku guys are particularly focused on mobile phones. Its not something I've tried yet because I think it would be a bit ex from where I live.

So Jaiku was the catalyst for me thinking about getting an RSS feed of my forum posts. Recently I've been trying to make an extra effort to contribute to the forums; frankly, they've always seemed a little quieter than I think they should be. So seeing any forums posts I make highlighted on Jaiku should be one of the neat indicators of my "web presence".

Problem is, while you can get a web page that lists your recent posts, and you can subscribe for email alerts when authors post, I wasn't able to find a way of getting an RSS feed for a specific author's posts.

So I created a little perl script that scrapes the HTML and generates an RSS feed (using XML::RSS::SimpleGen). I've packaged it as a CGI program on a server I have access to. That's what I registered on my experimental Jaiku site, and it works like a charm.

Until Oracle build this feature into the forums, feel free to take my oracleForumRSS.pl script and experiment away. Its pretty basic, but is generic for any forum user and ready to go. Sorry, but I'm not hosting it for direct use by others, so you'd have to find you own server with cgi or convert it to script that spits out a static rss xml page instead.

Post-script: Eddie Awad blogged on the "Easy Way" to do this using Dapper. Very cool, thanks Eddie!

Andon Tschauschev recently posted on perl.dbi.users concerning an apparent problem with temp tables "disappearing" between statements in the same session. He was using SQL Server via ODBC support.

The discussion and investigation continues, but it made me think to test if there's any similar strange behaviour with Oracle via DBI.

The temporary table model is somewhat different in Oracle, and centers around the "CREATE GLOBAL TEMPORARY TABLE.." statement. Temp table definitions are always global, but data is always private to the session, and whether data persists over a commit depends on whether the qualification "on commit preserve rows" or "on commit delete rows" is specified.

testOraTempTables.pl is a simple test script to check out the behaviour. Good news is that all seems to be a-ok. The temporary table definition is persistent across sessions, but data is not, and importantly (..the point of Andon's investigation..) data is preserved across DBI calls within the same session as expected.

Sample output from the test program:

C:\MyDocs\Testers2\perl\dbi>perl testOraTempTables.pl orcl scott tiger
[1st connection] connect to orcl {AutoCommit => 1}:
[1st connection] create global temp table:
create global temporary table t1 (x varchar2(10)) on commit preserve rows
[1st connection] insert 3 rows of data into it: insert into t1 values (?)
[1st connection] should be 3 rows because we have "on commit preserve rows" set:
select count(*) from t1 = 3
[2nd connection] connect to orcl:
[2nd connection] should be 0 rows because while the table definition is shared, the data is not:
select count(*) from t1 = 0
[2nd connection] disconnect:
[1st connection] disconnect:
[1st connection] reconnect {AutoCommit => 0}:
[1st connection] should be 0 rows because this is a new session:
select count(*) from t1 = 0
[1st connection] drop the temp table: drop table t1
[1st connection] create global temp table:
create global temporary table t1 (x varchar2(10)) on commit delete rows
[1st connection] insert 3 rows of data into it: insert into t1 values (?)
[1st connection] should be 3 rows because we have autocommit off and not committed yet:
select count(*) from t1 = 3
[1st connection] should be 0 rows because now we have committed:
select count(*) from t1 = 0
[1st connection] disconnect:
[1st connection] reconnect {AutoCommit => 1}:
[1st connection] insert 3 rows of data into it: insert into t1 values (?)
[1st connection] should be 0 rows because we have autocommit on and "on commit delete rows" defined:
select count(*) from t1 = 0
[1st connection] disconnect:
[1st connection] reconnect {AutoCommit => 0}:
[1st connection] drop the temp table: drop table t1
[1st connection] disconnect:

I've just released an update to tree-sync-2.2.pl on CPAN. This fixes a few bugs with special character handling in filenames.

I've currently three main uses for the script:

• On my Windows XP laptop, I use it to backup my working files to an external drive. For this I use the "fwdonly" mode so that the external drive copy is a perfect copy of what I have on the laptop.
• Also on my laptop, I have a collection of files that I sync with the external drive in "full" sync mode (default). This means changes will be exchanged bidirectionally. This is useful because sometimes I will drop files into this area with the external drive connected to another machine.
• Thirdly, I use it on my server to sync a collection of files from cvs into a web-visible area.

PS: As of Oct-2008, the tree-sync project is now on github. Use this if you want to contribute to development. Of course, releases will still be distributed for use on CPAN.

"Where's the alert log?" .. usually the first thing you want to know when looking at a new database.

Oracle's Grid Control solves this problem very well with it's web interface. But not always available.

Or in my case recently, where I had 6 databases setup on a single machine for various testing scenarios, I was getting tired of cd'ing all over the place, forgetting paths, or ending up with too many windows open for my own good. Getting well sick of this, I did a quick hunt for scripts to help but surprisingly didn't find much. So diving in, I created oraAlertLog.sh and now I'm happy;)

This script is for running on the database server. The Oracle environment must be set, and - at least for the first call - the database must be available so that the "background_dump_dest" parameter can be obtained. The script will cache the alert log location so that it will still work if the database happens to be down.

After getting this running I thought "duh!", should have been in perl so it would be possible to run on any platform supported by Oracle. Here's a version in Perl: oraAlertLog.pl. It requires Perl with DBI and DBD::Oracle .. the Perl distribution included with Oracle Database is fine for this (you just need to make sure the environment is properly configured).

Generating CDATA elements with Oracle XMLDB recently got a good airing in the XMLDB forums.

I won't reiterate the discussion there, but offer a summary and some sources.

It seems the current state of affairs is that if you need to generate large text elements with XMLDB you have two options:

• use DBMS_LOB procedural code to manually construct a CDATA element, or
  
• use XMLTYPE views to construct an XML-encoded element
  

In both cases you need to be careful not to do anything that casts or converts to varchar to avoid the inherent size limitations.

Note that the XML-encoding in XMLTYPE views is automatic, and I currently don't know how to tell it not to encode but rather quote as CDATA.

Some sources and examples:

• clob-cdata.pl is a Perl script using DBI that demonstrates how to generate an XMLTYPE view over an arbitrary CLOB element, without using XMLSchema. In this case, the CLOB will be automatically XML-encoded [clob-cdata-nonschema.sql is just the plain SQL].
  
• clob-cdata-schema.sql shows how you can do a similar thing, but using an XMLSchema definition.
  
• clob-cdata-small.sql shows how you can create CDATA elements where the text size is small using the XMLCdata function
  

Sometime back I was hacking my wifi admin pages (to let me register a certain NTP server .. but that's another story), and in the process discovered how broken the security is on my device (an SMC SMC2804WBRP-G Barricade router).

Basically the security check - to make sure you are a valid, logged-in administrator - just redirects to the "action" page which does no further checking of your credentials.

It doesn't take a genius to figure out that if you just post directly to the "action" page you can probably bypass authentication. At least, that's what occured to me, so I tried it and (too my surprise nonetheless) it worked. Or didn't work, depending on your point of view!

To their credit(!), the routine to reset the admin password does require you to send the existing password, but other operations have no barrier.

Here's a simple Perl script that demonstrates how you can "own" an SMC router of this type. It basically lets you reset factory defaults, after which you know the admin password (smcadmin). The factory default has no wifi enabled, so to make any further use of the router you must be connected to a LAN port. But certainly one way to wreck your neighbour's weekend.

I reported this vulnerability to SMC and CERT, but haven't heard whether any action has been taken to fix this.

I also don't know how many other models or brands of routers are susceptible to the same fault. But take this as a warning (and the reason why I am posting this information) ... if you want to offer wifi services to others, make sure your device is not subject to this kind of flaw first!

I've released an update to Chang Liu's tree-sync.pl script on CPAN (currently tree-sync-2.2.pl).

I had some problems with the original script, so this is a re-write that uses an algorithm based on File::Find module.

I've only been testing on Windows and Linux, but so far so good - it solves my immiedate issue which was to have a way of maintaining a synchronised "backup" copy of various filesystems.

PS: As of Oct-2008, the tree-sync project is now on github. Use this if you want to contribute to development. Of course, releases will still be distributed for use on CPAN.

Previously, I mentioned I'd come back to more complex request and response structures with SOAP::Lite.

Frankly, I haven't posted because I can't avoid the feeling that there's still a little more to unravel. Did I mention that good documentation is sparse? ;) Byrne and others have posted some good stuff (see for example the majordojo soaplite archive and this hands-on tour at builder.com), but mostly you'll find the experiences shared go along lines like "...after hacking around for a bit, I found that this worked...".

But is it possible to try and pin down at least a couple of guidelines for using SOAP::Data? Well, so far I can't claim to solving it all, but I am able to share a few anchors I've tried to plant for my own sanity!

My Rules for SOAP::Sanity

In the following "rules", $soap is a pre-initialised SOAP::Lite object, as in:
my $soap = SOAP::Lite->uri ( $serviceNs ) -> proxy ( $serviceUrl );

1. The value of a SOAP::Data element becomes the content of the XML entity.

It may seem bleeding obvious. Nevertheless, get this idea fixed in you head and it will help for more complex structures.

So if we are calling a "getHoroscope" service with the following request structure:

<getHoroscope>
<sign>Aries</sign>
</getHoroscope>

"Aries" is the value, i.e. the content, of the XML entity called "sign". Thus our request will look like this:

$data = SOAP::Data->name("sign" => 'Aries');
$som = $soap->getHoroscope( $data );

2. To create a hiearchy of entities, use references to a SOAP::Data structure.

In (1), the content of the entity was a simple string ("Aries"). Here we consider the case where we need the content to encapsulate more XML elements rather than just a string. For example a request with this structure:

<getHoroscope>
<astrology>
<sign>Aries</sign>
</astrology>
</getHoroscope>

Here "astrology" has an XML child element rather than a string value.
To achieve this, we set the value of the "astrology" element as a reference to the "sign" SOAP::Data object:

$data = SOAP::Data->name("astrology" =>
 \SOAP::Data->name("sign" => 'Aries') 
);
$som = $soap->getHoroscope( $data );

3. To handle multiple child entities, encapsuate as reference to a SOAP::Data collection.

In this case, we need our "astrology" element to have multiple children, for example:

<getHoroscope>
<astrology>
<sign>Aries</sign>
<sign>Pisces</sign>
</astrology>
</getHoroscope>

So a simple variation on (2). To achieve this, we collect the "Aries" and "Pisces" elements as a collection within an anonymous SOAP::Data object. We pass a reference to this object as the value of the "astrology" item.

$data = SOAP::Data->name("astrology" => 
 \SOAP::Data->value(
 SOAP::Data->name("sign" => 'Aries'), 
 SOAP::Data->name("sign" => 'Pisces') 
 )
);
$som = $soap->getHoroscope( $data ); 

4. Clearly distinguish method name structures from data.

This is perhaps just a style and clarity consideration. In the examples above, the method has been implicitly dispatched ("getHoroscope").
If you prefer (or need) to pass the method information to a SOAP::Lite call, I like to keep the method information distinct from the method data.

So for example, item (3) can be re-written (including some additional namespace handling) as:

$data = SOAP::Data->name("astrology" => 
 \SOAP::Data->value(
 SOAP::Data->name("sign" => 'Aries'), 
 SOAP::Data->name("sign" => 'Pisces') 
 )
);
$som = $soap->call( 
 SOAP::Data->name('x:getHoroscope')->attr({'xmlns:x' => $serviceNs})
 => $data 
);

I prefer to read this than have it all mangled together.

That brings me to the end of my list of rules! I am by no means confident that there aren't more useful guidelines to be added, or that in fact the ones I have proposed above will even stand the test of time.

Nevertheless, with these four ideas clearly in mind, I find I have a fair chance of sitting down to write a complex SOAP::Lite call correctly the first time, rather than the trial and error approach I used to be very familiar with!

SOAP::Lite is an incredibly powerful little module that makes using Perl for SOAP clients a piece of cake (groan).

True to the spirit of Perl, it makes simple things simple. This is all you need to do to interrogate the "Version" Axis (1+2) service:

#!/usr/bin/perl -w
use SOAP::Lite;
print SOAP::Lite
-> proxy('http://localhost/axis2/services/Version')
-> getVersion()
-> result;

The output:

Hello I am Axis2 version service , My version is 1.1

The SOAP request that this generates is as basic as it gets, and as you can see most of it is dedicated to the envelope definition:

<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope
xmlns:xsi=
"http://www.w3.org/1999/XMLSchema-instance"
xmlns:SOAP-ENC=
"http://schemas.xmlsoap.org/soap/encoding/"
xmlns:SOAP-ENV=
"http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsd=
"http://www.w3.org/1999/XMLSchema"
SOAP-ENV:encodingStyle=
"http://schemas.xmlsoap.org/soap/encoding/">
<SOAP-ENV:Body><getVersion/></SOAP-ENV:Body>
</SOAP-ENV:Envelope>

The example does illustrate one very important capability of SOAP::Lite - the automatic marshaling of the response into a usable Perl structure. In this case, its just a string.

One of the difficulties with SOAP::Lite at present is the dearth of good documentation and examples. This does tend to make doing more complex operations feel like you are living on the edge, but that's where the fun begins for some I guess;) I will look at some more complex request and response topics in a further post.