Its been a while

It has been a while since I last made a blog entry. Over the past couple of weeks I have been re-instating my web site and collecting the old blog entries and posting them again. I am now up to date with that, and free to continue with some entries that are fresh and new.

It has been a while since I last made a blog entry. Over the past couple of weeks I have been re-instating my web site and collecting the old blog entries and posting them again. I am now up to date with that, and free to continue with some entries that are fresh and new.

After a lot of difficult decisions, I have decided to repeat building my site using Drupal, something I was using before when it was a proper site. I am currently running this site on a Sheeva Plug with an SD card for the disk, and this is a little underpowered for the job it has to do and has also proved unreliable (the unit I purchased last November blew up with a power supply problem – and the word around is that I am not unique in finding this to be the case). I have found the machine “locking up” more than once – which normally doesn’t happen with linux.

So. I will shortly be reverting to a normal PC has my web server (once some ordered parts arrive and I have installed them) and whilst the power consumption will be greater (the reason for the Sheevaplug was to reduce power) I am afraid it is a necessity.

I still have to sort out problems with my photo albums, but I think I know how to deal with that and will be fixing the problems over the next week or so.

Providing a high speed connection between two web clients

I have been considering the architecture for a web based real time game between two players.  It doesn’t matter what the game is, any game in which there are two players.  I’m thinking of an air hockey simulation, but it could be anything (tennis maybe).  The basic idea is that each web browser will run a javascript/ajax application to control the player on their side, and will want to pass messages very rapidly to the other player.

I have been considering the architecture for a web based real time game between two players.  It doesn’t matter what the game is, any game in which there are two players.  I’m thinking of an air hockey simulation, but it could be anything (tennis maybe).  The basic idea is that each web browser will run a javascript/ajax application to control the player on their side, and will want to pass messages very rapidly to the other player.

My last application, a chat program, passed messages via a database, with the client polling the server every two seconds to see if anything new had arrived.  For a game this would not be fast enough. So I needed an alternative.  The obvious choice for passing data between two programs in unix is a pipe, and that is what I decided I would try and use.  But there are limitations.  I can pretty well only use php as a module under Apache web server for this activity.  That means that each pipe has to be opened, read or written to, and then closed in the space of one request.  Granted I can use ajax for that request to happen in parallel (ie not block the game), but how do you sync with the other client potentially on the other side of the world, who could go and walk away from the game at any time.

Continue reading “Providing a high speed connection between two web clients”

Next Steps with UserMgr

As the last article hinted, I had hoped to move forward with the web site. The next stage is an update to my UserMgr application.

As the last article hinted, I had hoped to move forward with the web site. The next stage is an update to my UserMgr application.

As you can see if you attempt to access certain areas of the web site you are required to provide a user name and password. Currently those usernames and passwords are stored in a database, and behind the (username and password protected) Site Administration tab there is an application that allows me to edit this database. Also, the Family Area is also protected and there is part of the application on this page that allows members of the family to change their own user information. The code from all this can be found in the Software Projects area.

Continue reading “Next Steps with UserMgr”

CSS Holy Grail

I just saw the article In Search of the Holy Grail about how to get a three column site to work reliably in CSS.

I tried some of the concepts in it on some static content, and it works really nicely – even if I only have a two column page and I set the right hand column width to 0.

So over the next few weeks, I am going to try and migrate my pages into this form. I have started internally to do this, and slowly you should see some alterations here as they get introduced.

I just saw the article In Search of the Holy Grail about how to get a three column site to work reliably in CSS.

I tried some of the concepts in it on some static content, and it works really nicely – even if I only have a two column page and I set the right hand column width to 0.

So over the next few weeks, I am going to try and migrate my pages into this form. I have started internally to do this, and slowly you should see some alterations here as they get introduced.

I am coupling that with a complete rething about how the border component should work with the tapestry apps on this site, because I would like to be able to submit forms via the menu in the blue menu area at the top, and at the moment I can’t. This will delay things a bit, because I have taken the border component to pieces and need to put it back together again:-)

Watch this space

My new Blog – with Lucene

Today is the day – my new blog has been made live

It has taken a while, because I got side tracked with getting my AKCMoney application working but I have completed my blog.

Today is the day – my new blog has been made live

It has taken a while, because I got side tracked with getting my AKCMoney application working but I have completed my blog.

I had been wanting to replace textpattern for a while, as I was unhappy with running two simulteneous database engines at the same time, and I had already decided to standardise on Postgres for my other applications. I did make a start on attempting to convert Textpattern over to Postgres, but once my experience of getting up to speed with Java took over, I wanted to use Java for my new blog.

The original idea was to just use a standard database (just as the old TextPattern application had) to store the articles, but I also needed to figure out a way to search the articles. It was then that I decided to try out lucence. The more I started to understand how to use it for searching, the more I realised that it would be just as good a tool to store the articles as well. So thats how I proceeded.

I also needed a “language” to write my articles in – Textpattern has Textile, and I needed something similar. As you can see in my article WYSIWYG v WYSIWYM we used to have a system based on nroff macros to make very high quality documentation. Using that as an inspiration, Roffer was born.

More use of interesting open source libraries, I built the language using JavaCC. and it seems to work very nicely indeed. All the articles in this site have been written using it. I made roffer act as the tokenizer in Lucene, so the combination of the two can act as a complete database system.

So I built a database layer, that is then used by the rest of the code to work with the basic concepts of articles, categories and some simple searching.

Using tapestry for the basic application completes the picture.

Source Code for the application will be released soon under the GPL.

UPDATE: 26th February 2011. The software has been retired some time ago, and I have since used WordPress, Drupal, and now back to WordPress for this software.  The original Java application is available in my old-apps repository

AKC Money is born

At work I got my new laptop last October. But I couldn’t get my Palm Pilot to Sync with it, so was getting increasingly worried that my data, particularly my diary, was getting out of date. Also, I had an application for managing my accounts (Micromoney) that contains absolutely critical financial data holding the state of my personal finances.

Near the end of November I decided to try and sync with firstly my Linux Desktop, and then when that didn’t work my old laptop. None of them worked anylonger. It looks as though something has gone wrong with the Palm.

At work I got my new laptop last October. But I couldn’t get my Palm Pilot to Sync with it, so was getting increasingly worried that my data, particularly my diary, was getting out of date. Also, I had an application for managing my accounts (Micromoney) that contains absolutely critical financial data holding the state of my personal finances.

Near the end of November I decided to try and sync with firstly my Linux Desktop, and then when that didn’t work my old laptop. None of them worked anylonger. It looks as though something has gone wrong with the Palm.

A bit of searching on the internet found that a possible cause is static charge buildup, but the only way out of this is to let the battery run down and destroy the contents.

So I decided on a different approach. Build an application to run on my web site that replicated the functionality. From a security point of view that required both login and https – so that I could get at the data from work when filling in my expenses, but I also felt I could improve the application in a few areas (such as the management of foreign currency) that the existing application was poor in.

So I have taken a step sideways and stopped working on my blog, to urgently build this new application.

Its taken about one month – which was longer than I had hoped, but its working. Of course I don’t want you all to see my financial data, so I also installed demo copy with its own database and not protected via login and https.

Things I added

  • The ability to work in Foreign Currencies for transactions. I must admit the way I have done this is slightly strange, but it matches my own workflow, in which I claim my expenses either as
    • the result of transactions where I have paid for cash (and therefore need to remember the exchange rate at the last converion, or
    • the result of credit card transactions where I will initially guess at the conversion rate using the same rate as the cash transactions, but which will be accurately reflected in actual GBP figures when the statement comes through
  • Items have to be separately cleared from both the source and destination accounts. My old version tried to implement this with a single cleared flag and it didn’t always work. I have two specific flags for each end of the transaction
  • recognise that some accounts normally have the majority of transactions as debit (e.g. my bank account where there is a single salary credit, and then lots and lots of debit transactions. On the other hand my Company Expenses account is a credit account (I credit it with debits from other accounts for every expense, its debited – and normally by bank acount is credited when the company repays me those expenses.

I will release source code soon (under the GPL) – as soon as I can get the site tidied up

Taking the Java Plunge – release 1.0 reached

Well I made it. It was early in the weekend that the code was working, but its taken a couple more days of struggle to get it so I could put it into production. Learning how to get tomcat to authorize the users is hell to debug when it doesn’t seem to do what the book says it should.

Looking back at what I have done surprises me. It was a simple application (just 5 web pages) but the number of separate technologies that I had to learn (or in some cases perfect) were many. Here is a list (on top of Debian Unstable/Stable)

Well I made it. It was early in the weekend that the code was working, but its taken a couple more days of struggle to get it so I could put it into production. Learning how to get tomcat to authorize the users is hell to debug when it doesn’t seem to do what the book says it should.

Looking back at what I have done surprises me. It was a simple application (just 5 web pages) but the number of separate technologies that I had to learn (or in some cases perfect) were many. Here is a list (on top of Debian Unstable/Stable)

  • download and conversion to debian package of sun sdk
  • set up and configuration of Tomcat
  • complilation of jk_mod from source
  • installation and configuration of jk_mod into Apache2
  • download, installation and use of Eclipse with Web Tools (including debugging)
  • download, installation and use of git (and cogito) scm system
  • Java Programming – with special attention having to be paid to the Collections classes
  • download, installation of Tapestry web framework
  • learning all of the Tapestry concepts (basic, forms with translaors and application state objects in particular)
  • download, installation of iBatis database framework
  • learning the use of iBatis concepts (with complex joins and how they populated Java collection classes)
  • set up of postgres access controls so that code and myself (using psql) could create and then access the database
  • SQL for key transactions, and in particular complex joins in order to extract the family relationships from the database
  • set up of jdbc driver for Postgres and configuration in Tomcat so that could access family tree database
  • set up of a datasource realm in Tomcat and a users database in Postgres to provide access controls to application
  • html encoding
  • css encoding
  • javascript and how to use within the Tapestry framework
  • Povray to create the chandlerfamily logo (not in the timeframe of this project – was done earlier)
  • inkscape to draw graphic items
  • gimp to convert graphic items to gif

Taking the JAVA plunge – into the last mile

After some hard work at getting my ideas together and then some marathon debugging sessions, my application is almost working. It has taken me a lot longer than I envisioned – for two real reasons.

Firstly, for a long time I had an approximate idea of what screens and what workflow I wanted in the application but I was finding it difficult to make it concrete. When I didn’t have a clear picture of what I was aiming at, my development didn’t proceed.

After some hard work at getting my ideas together and then some marathon debugging sessions, my application is almost working. It has taken me a lot longer than I envisioned – for two real reasons.

Firstly, for a long time I had an approximate idea of what screens and what workflow I wanted in the application but I was finding it difficult to make it concrete. When I didn’t have a clear picture of what I was aiming at, my development didn’t proceed.

In the end, I found that to draw out the screen layouts and then merge them onto a single page as the best thing to do. I then printed them out and annotated the copies with the actions and how they were going to control variables across the application. This picture shows the layout that I drew (which is close to what I will end up with).

Unfortunately the picture that I had placed here in this blog entry has been lost

I found Inkscape www.inkscape.org under both Windows and Linux as a really good tool to do this. I first drew each page as a separate drawing, and then used the import tool to pull them all together in a single drawing.

Unfortunately, I found Inkscape cumbersome for drawing arrows to represent the transitions between the pages. Initially I just used marks on the printed copies, but these got difficult to see with so many scribbles, so in the end I used KIVIO under kde to draw single simple boxes for the pages and coloured arrows for the flows between them.

update I now use DIA for this, and export the result as an SVG which I can then size and export as a PNG image using Inkscape

The second reason for lack of progress was trying to be too clever with not creating a session objects to store persistance data in. I was trying to carry the data between pages, and getting screwed up when the page was being recreated without access to the information I wanted. As soon as I created a few Tapesty “Application State Objects” for my session, and just got standard forms to load data to and from these objects everything became a lot simpler.

Now just trying to complete the javascript for a dynamic piece on my Edit page, looking at using CSS for control of the formats on all pages (and using the same to provide mouse hover tool tips) and I will be moving on to production deployment.

I’ll report on that soon.