Thursday, December 3, 2009

How does SSL work?

How Does SSL Work?


Every modern web browser has built into it a very sophisticated encryption system that allows secure "conversations" with webservers equipped with an SSL secure server (See endnote).

This is about to get pretty technical so I am simplifying it greatly. If you find your eyes glazing over, don't worry. The main message here is that SSL ensures an encrypted communication between the customer's web browser and the merchant's webserver. This produces both customer confidence and a very secure communication system. Use it!

But some of you want to know how it works, so here goes.

Keys and Identities

You remember the Decoder Ring you found in a cereal box when you were a kid? It consisted of a wheel with the alphabet on each of two rotating discs. If you someone sent you a message where the "key" is A=L, you'd set the A above the L, and then determine that B=M and C=N, etc. Modern cryptography uses a very complex algorithm or formula to encode messages using a key. Rather than A=L, the key will be very large number used to scramble the text. If you were to give the key to your friend, he would be able to decode the messages you encode with it and visa versa.

Two problems present themselves, however, when you don't have face-to-face contact with a trusted friend.

  • Who is your friend and who is the evil hacker? The Internet is a anonymous place where people are not always whom they seem. If you've spent any time in chat rooms, you know that "Pixiebabe" just might be a middle-aged man in Toledo. How do you know that you're speaking to the real online store, or just a mimic of it? That's where a Digital Certificate comes in, to identify the store as authentic, verified by a trusted Third Party (called the Certificate Authority).
  • In addition, on the Internet you can't physically hand over a key, and whatever key you send electronically could be intercepted by an evil hacker. Enter a complex technology called Public Key encryption.

Private Keys and Public Keys

The way SSL solves this problem is to have two keys: a private key (that you never disclose to anyone) and a public key (which you may distribute freely to the general public). This is how it works: You send your friend your public key and he uses it to send you an encrypted message that only you can decode when you use your private key. Your friend can only encode the message; he wouldn't be able to decode it, since he only has your public key, not your private key. Confused yet?

You may be, but your web browser can keep it straight, and comes with the ability to encode and decode messages sent to and from an SSL-secure webserver.

Getting to Know You

The merchant's in-house webserver (or the merchant's web hosting service) has a Digital Certificate or Digital ID that identifies it as the authentic owner of a certain domain name. The Digital Certificate contains the several pieces of information, including the merchant's public key, signed by the Certificate Authority (such as VeriSign).

The customer's web browser has built into it a list of recognized Certificate Authorities (such as VeriSign) and their unforgeable digital signatures, so it can verify if a merchant's Digital Certificate is valid or a forgery.

Now the customer's web browser and the merchant's website begin a getting-to-know-you exchange, kind of like fax machines squawking as they're getting in synch with each other. Here are some of the important steps in this mating ritual:

  1. The customer's web browser requests a secure page, such as https://www.yourcompany/orderform.htm
  2. The merchant's webserver responds by sending the customer's web browser the Digital Certificate containing all its information.
  3. The customer's web browser checks to be sure that the Digital Certificate is valid, hasn't expired, and matches the Certificate's domain name and the list of digital signature of Certificate Authorities the web browser has on file. Then it replies by encrypting a message using the merchant's public key. In this message will be a unique Session Key generated by the web browser from a random number.
  4. The merchant's webserver decodes the unique Session Key using the merchant's Private Key. Now both the customer and the merchant have the same unique Session Key. If you were an evil hacker listening in, you could hear both the messages from the webserver and the customer, but you couldn't decode the customer's message to the webserver, since you don't possess the merchant's Private Key. (I'm skipping over some other checks performed at this time.)

    Now the merchant's webserver uses the unique Session Key to encode a message to customer's webserver, and visa versa. The two are in step and the dance has begun, dancing to a tune that only they can hear. They can whisper sweet nothings to each other encoded with this unique Session Key, and no one can tell what they're saying.

So long as the evil hacker can't guess the unique Session Key, he can't decode the messages being passed back and forth. It is gobbldy-gook so far as he is concerned. Now the customer can give his credit card number safely and the web server can echo it back for verification without any chance of an evil hacker kidnapping it on the way by.

That's how SSL works. Actually that's the kindergarten version of an extremely complex subject. But I hope it's enough to give you some confidence in the process. It really works quite smoothly and securely.

Note: SSL stands for "Secure Sockets Layer." "Socket" is a technical term that refers to an application programming interface, or API, which is used to communicate between two computers. "Layer" refers to the level or layer of this communication between the computers.

Thursday, November 12, 2009

What is wrong with BufferedOutputStream

I was writing a stream into a file. I initialized a BufferedOutputStream with new BufferedOutputStream(new FileOutputSteam("filename");

And I closed all input and output stream after job is done. But the file is always corrupted, there's about 4k missing from the original input stream.

But after I removed the BufferedOutputStream, the problem solved. Why is that?

Wednesday, November 4, 2009

Why shouldn't you hire Maureen Wood as home inspector?

Maureen Wood is the home inspector from Town & Country Home Inspections. She is the worst home inspector ever. There are many reasons support this.

1) She only check roofing from ground level, that means she doesn't check gutter, roof surface. If your house have chimney, then the connection between roof and chimney is also not able to inspect from ground level.

2) She doesn't inspect attic by entering into the attic. She only puts half of her head into attic, and take some pictures of the ceiling of the attic. That means she won't inspect the depth of the insulation in the attic. She won't check the floor of the attic and that could hide a lot of secretes, like raccoon problem. And she also won't check if soffit vent is open to the attic.

3) She has no home construction experience and she is not handy as well. She couldn't access attic because she can't remove the sliding door from closet where the attic entrance is. Some common problems in a house which should be a simple maintain job for home owner like hair clog in the drain. She identified as a problem rather than gives home buyer suggestion how to maintain it.

4) She duplicates paragraphs in the home inspection report in order to make the report looks thick, because she knows nobody will read the report thoroughly.

5) She knows very well to protect herself. If she failed her job, home buyer is very hard to sue her.

Thursday, October 8, 2009

Fix "Related To" problem in Sugar Notes module

We have a customized module "Installation" in Sugar. It related to Notes module. For some reason, when user create a note from Installation module, it doesn't automatically pull out the related Installation record. As a matter of fact, in "related to" field, "Installation" is not even an option.

I know a Dropdown named "parent_type_display", I add Installation into there, but it works for any other activities, such as Call, Meeting.. except Notes.

Then I decided to trace it from the code. I started from index.php, then I found out it is a long trip to reach the problem. I only care about final display, so if the dropdown options is not from "parent_type_display", then it must be somewhere to generate the select options. I need to find how the final html is generated. Then I find file include/EditView/EditView2.php, and it has a method display(). I believe this should be the last method to generate the html stream. I put a log right before the statement
$str .= $this->th->displayTemplate($this->module, $this->view, $this->tpl, $ajaxSave, $this->defs);

Then assumption is approved. This string is the final html. Then I went a step further, dig into method displayTemplate() in file include/TemplateHandler/TemplateHandler.php.

Hmm, it's the place to find what is the final template file. I put a log statement right after :
$file = $this->cacheDir . $this->templateDir . $module . '/' . $view . '.tpl';
$GLOBAL['log']->debug("template file name : ".$file);

I found out the template file is catch/modules/Notes/editview.tpl. Then I open this file, search for "select". found this:
{html_options options=$fields.parent_name.options selected=$fields.parent_type.value}

I know fields is an array defined in modules/Notes/vardefs.php, open that file, I see this:
 'parent_name'={
     array(
        'name'=> 'parent_name',
        'parent_type'=>'record_type_display' ,
        'type_name'=>'parent_type',
        'id_name'=>'parent_id', 'vname'=>'LBL_RELATED_TO',
        'type'=>'parent',
        'source'=>'non-db',
        'options'=> 'record_type_display_notes',
        ),

Here it is, the dropdown is not "parent_type_display", I check other module's definition, such as "meetings, calls", they all use "parent_type_display". That's why change this dropdown values, "related to" can be changed for other module except Notes.

Then I goto Dropdown Editor, add my customized module name into "record_type_display_notes". Problem solved.


Tuesday, October 6, 2009

The worst home inspector(1)

I have hired Maureen Wood of Town & Country Home Inspections to inspector my semi-detached house this summer. I interviewed her over the phone, and specially asked if she climbs up to the roof to check roofing. She answered positively.


But she just looked the roof from ground level, and she didn’t discover a hole on the side of the chimney which is a hole torn by raccoon. In the first day I moved in to that house, I discovered a family of raccoon lived in the attic.


If she looked the roof from eaves, she should be able to discover that hole. Or if she was a person who is attention to detail, she should be able to find the mess left by raccoon.   She didn’t enter the attic, just raised her hand into the attic and took a couple of pictures from the attic entrance hole. Even that, she refused to admit she did a poor job and refuse to admit any liability.


If a home inspector didn’t have home construction experience, he or she should at least very handy. After I read the home inspection report thoroughly, I realize this home inspector did nothing for me.


Anything is not visible to me; she has no idea as well. She doesn’t have any device can detect the hazard material, such as asbestos, radon, lead and UFFI.


Something like faucet drips, sink drain clogged… oh, come on, I can see it myself. And home inspector is supposed to give home buyer some advices to fix those problems. Those are all minor problems; anybody who is a little bit handy can fix those themselves.


For example, the clogged drain, there’s a cap on the drain pipe under the sink, if you remove that cap, you can take out the drainage blocker, and clean it regularly. But I believe not everyone knows that, at least the former home owner didn’t know that. And my home inspector probably didn’t know that as well. Because she just told me the drain is clogged, didn’t say anything that is an easy fix.


I have owned my first house for a month. One advice to new home owner is that never hire a home inspector who doesn’t have home construction experience and not able to check roof and attic.

Wednesday, September 23, 2009

Recover mysql root password

Recover MySQL root password

You can recover MySQL database server password with following five easy steps.

Step # 1: Stop the MySQL server process.

Step # 2: Start the MySQL (mysqld) server/daemon process with the --skip-grant-tables option so that it will not prompt for password

Step # 3: Connect to mysql server as the root user

Step # 4: Setup new root password

Step # 5: Exit and restart MySQL server

Here are commands you need to type for each step (login as the root user):

Step # 1 : Stop mysql service

# /etc/init.d/mysql stop
Output:

Stopping MySQL database server: mysqld.

Step # 2: Start to MySQL server w/o password:

# mysqld_safe --skip-grant-tables &
Output:

[1] 5988
Starting mysqld daemon with databases from /var/lib/mysql
mysqld_safe[6025]: started

Step # 3: Connect to mysql server using mysql client:

# mysql -u root
Output:

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1 to server version: 4.1.15-Debian_1-log

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql>

Step # 4: Setup new MySQL root user password

mysql> use mysql;
mysql> update user set password=PASSWORD("NEW-ROOT-PASSWORD") where User='root';
mysql> flush privileges;
mysql> quit

Step # 5: Stop MySQL Server:

# /etc/init.d/mysql stop
Output:

Stopping MySQL database server: mysqld
STOPPING server from pid file /var/run/mysqld/mysqld.pid
mysqld_safe[6186]: ended

[1]+ Done mysqld_safe --skip-grant-tables

Step # 6: Start MySQL server and test it

# /etc/init.d/mysql start
# mysql -u root -p

Thursday, September 17, 2009

All about regular expression

Javascript Regular Expression

Difference with Java regular expression
1. Flags
    JavaScript uses / follows (i, g, m) and the end of the pattern string.
    Java put the flag in the beginning of the pattern string.

  
JavaScript:   /\bjava\b/i
   Java: "(?i)\bjava\b"




Java Regular Expression


1. Differences Among Greedy, Reluctant, and Possessive Quantifiers

There are subtle differences among greedy, reluctant, and possessive quantifiers.

Greedy quantifiers are considered "greedy" because they force the matcher to read in, or eat, the entire input string prior to attempting the first match. If the first match attempt (the entire input string) fails, the matcher backs off the input string by one character and tries again, repeating the process until a match is found or there are no more characters left to back off from. Depending on the quantifier used in the expression, the last thing it will try matching against is 1 or 0 characters.

The reluctant quantifiers, however, take the opposite approach: They start at the beginning of the input string, then reluctantly eat one character at a time looking for a match. The last thing they try is the entire input string.

Finally, the possessive quantifiers always eat the entire input string, trying once (and only once) for a match. Unlike the greedy quantifiers, possessive quantifiers never back off, even if doing so would allow the overall match to succeed.

To illustrate, consider the input string xfooxxxxxxfoo.

 
Enter your regex: .*foo // greedy quantifier
Enter input string to search: xfooxxxxxxfoo
I found the text "xfooxxxxxxfoo" starting at index 0 and ending at index 13.

Enter your regex: .*?foo // reluctant quantifier
Enter input string to search: xfooxxxxxxfoo
I found the text "xfoo" starting at index 0 and ending at index 4.
I found the text "xxxxxxfoo" starting at index 4 and ending at index 13.

Enter your regex: .*+foo // possessive quantifier
Enter input string to search: xfooxxxxxxfoo
No match found.
The first example uses the greedy quantifier .* to find "anything", zero or more times, followed by the letters "f" "o" "o". Because the quantifier is greedy, the .* portion of the expression first eats the entire input string. At this point, the overall expression cannot succeed, because the last three letters ("f" "o" "o") have already been consumed. So the matcher slowly backs off one letter at a time until the rightmost occurrence of "foo" has been regurgitated, at which point the match succeeds and the search ends.

The second example, however, is reluctant, so it starts by first consuming "nothing". Because "foo" doesn't appear at the beginning of the string, it's forced to swallow the first letter (an "x"), which triggers the first match at 0 and 4. Our test harness continues the process until the input string is exhausted. It finds another match at 4 and 13.

The third example fails to find a match because the quantifier is possessive. In this case, the entire input string is consumed by .*+, leaving nothing left over to satisfy the "foo" at the end of the expression. Use a possessive quantifier for situations where you want to seize all of something without ever backing off; it will outperform the equivalent greedy quantifier in cases where the match is not immediately found.

2. Capturing Groups
In the previous section, we saw how quantifiers attach to one character, character class, or capturing group at a time. But until now, we have not discussed the notion of capturing groups in any detail.

Capturing groups are a way to treat multiple characters as a single unit. They are created by placing the characters to be grouped inside a set of parentheses. For example, the regular expression (dog) creates a single group containing the letters "d" "o" and "g". The portion of the input string that matches the capturing group will be saved in memory for later recall via backreferences (as discussed below in the section, Backreferences).

Numbering (The order count from outside to inside, then left to right)

As described in the Pattern API, capturing groups are numbered by counting their opening parentheses from left to right. In the expression ((A)(B(C))), for example, there are four such groups:
  1. ((A)(B(C)))
  2. (A)
  3. (B(C))
  4. (C)
To find out how many groups are present in the expression, call the groupCount method on a matcher object. The groupCount method returns an int showing the number of capturing groups present in the matcher's pattern. In this example, groupCount would return the number 4, showing that the pattern contains 4 capturing groups.

There is also a special group, group 0, which always represents the entire expression. This group is not included in the total reported by groupCount. Groups beginning with (? are pure, non-capturing groups that do not capture text and do not count towards the group total. (You'll see examples of non-capturing groups later in the section Methods of the Pattern Class.)

It's important to understand how groups are numbered because some Matcher methods accept an int specifying a particular group number as a parameter:

Wednesday, September 9, 2009

Fix seLinux problem

Q:

I have some avc denials that I would like to allow, how do I do this?

A:

If you have specific AVC messages you can use audit2allow to generate a Type Enforcement file that is ready to load as a policy module.

audit2allow -M local < /tmp/avcs

This creates a local.pp which you can then load into the kernel using semodule -i local.pp. You can also edit the local.te to make additional customizations. To create a module allowing all the denials since the last reboot that you can then customize, execute the following:

audit2allow -m local -l -i /var/log/messages > local.te

Note that the above assumes you are not using the audit daemon. If you were using the audit daemon, then you should use /var/log/audit/audit.log instead of /var/log/messages as your log file. This generates a local.te file, that looks similar to the following:

module local 1.0;

require {
class file { append execute execute_no_trans getattr ioctl read write };
type httpd_t;
type httpd_w3c_script_exec_t;
};


allow httpd_t httpd_w3c_script_exec_t:file { execute execute_no_trans getattr ioctl read };

You can hand edit this file, removing allow statements that you don't want to allow, and then recompile and reload it using

  • checkmodule -M -m -o local.mod local.te to compile the te file. Note that checkmodule is part of the checkpolicy rpm, so you need to have it installed.

  • semodule_package -o local.pp -m local.mod to create a policy package.

  • semodule -i local.pp to add it to the current machine's running policy. This installs a new module called local with these rules into the module store.

[Note] Important

In order to load this newly created policy package into the kernel, you are required to execute semodule -i local.pp

Note that if you later install another module called local, it will replace this module. If you want to keep these rules around, then you either need to append future customizations to this local.te, or give future customizations a differe




Tuesday, July 21, 2009

My first perl script

This is a perl script to scan a list of folders with certain name pattern, and retrieve certain information from specific file, then put into a CSV file and send a email to user with attachment.


#!/usr/bin/perl -w
#fine file name list and put into an array
use MIME::Lite;
sub findFile($);

sub sendEmail
{
my ($to, $from, $subject, $message) = @_;
my $sendmail = '/usr/sbin/sendmail';
open(MAIL, "|$sendmail -oi -t");
print MAIL "From: $from\n";
print MAIL "To: $to\n";
print MAIL "Subject: $subject\n\n";
print MAIL "$message\n";
close(MAIL);
}

sub findFile($)
{
my $dirname = $_[0];
my @matches = ();
local *DIR;
opendir ( DIR, $dirname ) || print "failed to open dir";
while( ($filename = readdir(DIR))){
$entry = "$dirname/$filename";
if (-d $entry )
{
if ($filename =~ m/INBOX/ || $filename =~ m/^\d\d\d\d$/)
{
push @matches, findFile($entry);
}
}
if (-f $entry && $filename =~ m/^msg\d\d\d\d.txt$/)
{
# print"filename ==============","$filename\n";
push @matches, $entry;
}
}
closedir(DIR);
return @matches;
}

@file_list = findFile("/var/lib/asterisk/sounds/voicemail/default");
# print @file_list;
#open an output file
#$out_file = '/tmp/tmp_file'; #define the name of output file
#open(OUT_FILE, ">$out_file");

$output_str = '';
#put head
#print OUT_FILE "Origbox, CallerId, OrigDate, Duration\n";
$output_str = $output_str."Origbox, CallerId, OrigDate, Duration\n";

#loop through file list
foreach $in_file_name (@file_list)
{
#open input file
open(IN_FILE, $in_file_name);
@lines = <IN_FILE>; #read file into an array

#loop through lines
%props = ();
foreach $_ (@lines)
{
if (/^origmailbox=/)
{
@entry = split(/=/, $_);
chomp($entry[1]);
%props = (%props, origmailbox=>$entry[1]);
}
if (/^callerid=/)
{
@entry = split(/=/, $_);
chomp($entry[1]);
%props = (%props, callerid=>$entry[1]);
}
if (/^origdate=/)
{
@entry = split(/=/, $_);
chomp($entry[1]);
%props = (%props, origdate=>$entry[1]);
}
if (/^duration=/)
{
@entry = split(/=/, $_);
chomp($entry[1]);
%props = (%props, duration=>$entry[1]);
}

}
if (length($props{"origmailbox"}) > 0) {
#print OUT_FILE "$props{origmailbox},$props{callerid},$props{origdate},$props{duration}\n";
$output_str = $output_str."$props{origmailbox},$props{callerid},$props{origdate},$props{duration}\n";
}

#finished job and close input file
close(IN_FILE);
}
#close(OUT_FILE);

my $msg = MIME::Lite->new(
From => 'dev@abc.com',
To => 'tom@abc.com',
Cc => 'jerry@abc.com',
Subject => 'Fonality Voice Mail Box Statistics',
Type => 'multipart/mixed',
);

$msg->attach(
Type => 'TEXT',
Data => "Please see attached file for detail...",
);
$msg->attach(
Type => 'text/csv',
Data => $output_str,
Filename => "fonality.csv",
);

$msg->send;

A shell script to do a bunch of things

This is a shell script I wrote for Krim to do following things:
1. Add a mount point into /etc/fatab.
2. create user and mount the folder.


#!/bin/bash

# add lab mount point into /etc/fstab. then create user, mount the folder
# Usage: addlab <vlab1|vlab2> <user_number>
# eg: addlab vlab1 8978
if [ $# -eq 0 ]
then
echo "Usage: $0 <vlab1|vlab2> <user_number>"
exit -1
fi

echo "You are about to create a lab for user lifelabs_$2 [y/n]:"; read x

if [ $x != 'y' ]
then
echo "bye"
exit -1
fi

vlab1="//to-vlab1/LIFELABS_$2 /home/lifelabs_$2/data cifs rw,mand,file_mode=0666,gid=lifelabs_$2,uid=lifelabs_$2,nosetuids,user=TO-VLAB1/lifelabs,password=velvet 0 0"

vlab2="//to-vlab2/LIFELABS_$2 /home/lifelabs_$2/data cifs rw,mand,file_mode=0666,gid=lifelabs_$2,uid=lifelabs_$2,nosetuids,user=TO-VLAB2/lifelabs,password=velvet 0 0"


#search if this lab already in fstab
if grep -q LIFELABS_$2 /etc/fstab
then
echo "LIFELABS_$2 IS ALREADY EXIST IN FSTAB FILE !!! EXIT"
exit -1
fi

if [ "vlab1" == $1 ]
then
echo $vlab1
echo "$vlab1" >> /etc/fstab
elif [ $1 == "vlab2" ]
then
echo $vlab2
echo "$vlab2" >> /etc/fstab
else
echo "no such lab - $1"
exit -1
fi

#add a user and setup password
adduser lifelabs_$2
echo lifelabs_$2:LIFELABS$2 | chpasswd

su - lifelabs_$2 -c "mkdir data"

mount /home/lifelabs_$2/data

exit 0

Tuesday, June 30, 2009

Difference between getCurrentSession() and openSession() in Hibernate ?

getCurrentSession() :
The "current session" refers to a Hibernate Session bound by Hibernate behind the scenes, to the transaction scope.
A Session is opened when getCurrentSession() is called for the first time and closed when the transaction ends.
It is also flushed automatically before the transaction commits. You can call getCurrentSession() as often and anywhere you want as long as the transaction runs.
To enable this strategy in your Hibernate configuration:

set hibernate.transaction.manager_lookup_class to a lookup strategy for your JEE container
set hibernate.transaction.factory_class to org.hibernate.transaction.JTATransactionFactory

Only the Session that you obtained with sf.getCurrentSession() is flushed and closed automatically.

Example :
try {
    UserTransaction tx = (UserTransaction)new InitialContext().lookup("java:comp/UserTransaction");

    tx.begin();

    // Do some work
    sf.getCurrentSession().createQuery(...);
    sf.getCurrentSession().persist(...);

    tx.commit();
}
catch (RuntimeException e) {
    tx.rollback();
    throw e; // or display error message
}

openSession() :
If you decide to use manage the Session yourself the go for sf.openSession() , you have to flush() and close() it.
It does not flush and close() automatically.
Example :
    UserTransaction tx = (UserTransaction)new InitialContext().lookup("java:comp/UserTransaction");

    Session session = factory.openSession();

    try {
        tx.begin();

         // Do some work
         session.createQuery(...);
         session.persist(...);

         session.flush(); // Extra work you need to do

         tx.commit();
    }
    catch (RuntimeException e) {
        tx.rollback();
        throw e; // or display error message
    }
    finally {
        session.close(); // Extra work you need to do
    }

Thursday, June 25, 2009

Struts ActionMessages and ActionErrors

Tutorials : Handling Messages, Errors and Exceptions in Struts 1.1 :

by Keld H. Hansen

Introduction

This article is about Jakarta Struts 1.1, the most popular framework for building Java servlet applications. One of the new, useful features in 1.1 is the possibility to specify exception handling in the Struts config file. It's called "declarative exception handling", and it's one of the topics for this article.

You use exceptions when the application has encountered a serious error. The error could be an SQL statement that fails, or a connection to another computer that couldn't be established. As a programmer, you should first of all record the situation as precisely as you can. For example, writing information to a log file. You might also have to fix up a few things to keep the program state stable, but eventually you'll also have to inform the user of the situation. So to move forward, one issue in exception handling is how to handle messages, and I'll therefore start by explaining how you generally should handle messages for the end user in Struts. A special class of messages are "validation messages", which are sent out as the result of the validation of a form. If you're interested in how to set up forms validation you may want to read my article "Stepping through the Struts 1.1 Validator".

After having discussed message handling in Struts, we'll see how Struts handles exceptions and start to investigate what the benefits of declarative exception handling could be.

Handling of Messages

Let's set the scene: we're in the Action class and want to send a message to the user at the browser. It could be anything from a kind message like "Your data has been stored successfully in the data base", or a warning about some incorrect data entered in a form, to the real bad news like "Database failure. Please contact support". Since the message is going to be displayed in a browser, it will have to be inserted in some way in a jsp-page.

What if we had to invent a message handling scheme ourselves? A sensible solution could be to put our messages in some kind of Java Collection, maybe a LinkedList:

Listing 1: Using a List to hold messages

List messages = new LinkedList();
messages.add("Data saved successfully");
messages.add("Please continue");
request.setAttribute("messages", messages);

In the jsp-page we could then easily show the messages using Struts tags:

Listing 2: Displaying the messages in a jsp-page

<logic:present name="messages">

<h3>Messages:</h3>

<logic:iterate id="msg" name="messages">
<bean:write name="msg"/><br>
</logic:iterate>

When the jsp-page displays we'll see this:

Messages:

Data saved successfully 
Please continue 

No rocket science in this. However, Struts has a more powerful mechanism for storing and handling messages based upon the ActionMessage and ActionMessages classes. If you're already a Struts user you may know the ActionErrors and ActionError classes. They're almost identical to the Message classes, and most of what I'll explain below for the Message classes works equally well for the Error classes. How we cope with having two different set of classes I'll return to later.

Using ActionMessages

The ActionMessages class resembles the List class we used above in that it is a container for the messages. The messages themselves are held in instances of the ActionMessage class:

Here's a piece of code from an Action class that uses these classes:

Listing 3: Using ActionMessages

ActionMessages messages = new ActionMessages();
ActionMessage msg = new ActionMessage("data.ok");
messages.add("message1", msg);

msg = new ActionMessage("data.continue");
messages.add("message2", msg);

saveMessages(request, messages);

As you can see there are some differences when you compare this to Listing 1:

  1. You don't give the message text directly to ActionMessage. Instead you give a key to the Struts message resource file (also called "the application property file"). I've specified these two lines in this file:
        data.ok=Data saved successfully data.continue=Please
    continue
    It's considered good practice to keep your fixed texts in the message resource file, and it also gives you the possibility to support several languages in your application. 
  2. You give a "label" when you add an ActionMessage. This makes it possible to look up the message and stick it in your HTML wherever you like it. We'll see below how to do this.
    It's possible to have more than one message with a given label.
    If the message is not related to anything special you use the label ActionErrors.GLOBAL_ERROR.

To show all the messages in the browser you add these lines to your jsp-page (note the resemblance with the code from listing 2):

Listing 4: The messages tag in use

<html:messages id="msg" message="true">
<bean:write name="msg"/><br>
</html:messages>

This will give you exactly the same output as above:

Messages:

Data saved successfully 
Please continue 

As you can see the <html:messages> tag loops over all the messages in the ActionMessages class. If you want to pick a single message from the class you specify its label as the property:

<html:messages id="msg2" message="true" property="message1">
<bean:write name="msg2"/><br>
</html:messages>

This makes it possible to place messages, for example, next to input fields in a form.

One more thing about the ActionMessage class: you may specify parameters to the message taken from the message resource file:

msg = new ActionMessage("data.do", "stop");
messages.add("message2", msg);

With "data.do=Please {0}" this will display as "Please stop".

Looking inside the ActionMessage(s) classes

You've seen how to put data inside the ActionMessage and ActionMessages classes, but it's also possible to pull data out again without having to use the Struts tags. You'd want to do this if you had to write your own tags for displaying messages. Here, first, are the methods you'd have to know from the classes:

Class Method Purpose
ActionMessages Iterator properties() Get the "labels" you used when storing the ActionMessage instances
-"- Iterator get(String property) Get the ActionMessage objects 
ActionMessage String getKey() Get the message key for this message
-"- Object[] getValues() Get the replacement values for this message 

You'll also need to know how to find the ActionMessages object. Struts stores it in the request object using the string value "org.apache.struts.action.ACTION_MESSAGE". This value should not be hard coded, but taken from the org.apache.struts.Globals class, which contains most of the keys used for storing various objects. We can also find a pointer to the message resources in the Globals class, and if we want to handle several languages we should also pull out the "locale".

To make it simple here's some code for a jsp-page that peeks into the ActionMessage objects:

Listing 5: The inspectmessages.jsp page

 . .
<%@ page import="java.util.*" %>
<%@ page import="org.apache.struts.*" %>
<%@ page import="org.apache.struts.util.*" %>
<%@ page import="org.apache.struts.action.*" %>

<%
// Print all attributes in the request object
out.println("<p><b>All Attributes in request scope:</b>");
Enumeration paramNames = request.getAttributeNames();
while (paramNames.hasMoreElements()) {
String name = (String) paramNames.nextElement();
Object values = request.getAttribute(name);
out.println("<br> " + name + ":" + values);
}

// Print all attributes in the session object
out.println("<p><b>All Attributes in session scope:</b>");
paramNames = session.getAttributeNames();
while (paramNames.hasMoreElements()) {
String name = (String) paramNames.nextElement();
Object values = session.getAttribute(name);
out.println("<br> " + name + ":" + values);
}

out.println("<p><b>Data in ActionMessages:</b>");

// Get the ActionMessages
Object o = request.getAttribute(Globals.MESSAGE_KEY);
if (o != null) {
ActionMessages ae = (ActionMessages)o;

// Get the locale and message resources bundle
Locale locale =
(Locale)session.getAttribute(Globals.LOCALE_KEY);
MessageResources messages =
(MessageResources)request.getAttribute
(Globals.MESSAGES_KEY);

// Loop thru all the labels in the ActionMessage's
for (Iterator i = ae.properties(); i.hasNext();) {
String property = (String)i.next();
out.println("<br>property " + property + ": ");

// Get all messages for this label
for (Iterator it = ae.get(property); it.hasNext();) {
ActionMessage a = (ActionMessage)it.next();
String key = a.getKey();
Object[] values = a.getValues();
out.println(" [key=" + key +
", message=" +
messages.getMessage(locale,key,values) +
"]");
}
}
}
%>
. . .

As you can see, I also print out the objects in the request and session objects. If we have the same data in the ActionMessages object as in Listing 3 we'll get this output from the jsp-page:

All Attributes in request scope: 
. . . (a few lines left out here) . . .
org.apache.struts.action.MESSAGE:
   org.apache.struts.util.PropertyMessageResources@a336d5 
org.apache.struts.action.ACTION_MESSAGE:
   org.apache.struts.action.ActionMessages@3bedc4 

All Attributes in session scope: 
org.apache.struts.action.LOCALE:en 

Data in ActionMessages: 
property message2: [key=data.continue, message=Please continue] 
property message1: [key=data.ok, message=Data saved successfully] 

The two last lines show the same messages as we got from the code in listing 3. At the top you can also identify the request keys for the ActionMessages object and the message resource bundle.

The old ActionError(s)

If you've been using Struts before version 1.1 you probably know the ActionErrors and ActionError classes and the corresponding <html:errors> tag. The new ActionMessages and ActionMessage classes and <html:messages> tag are meant to replace these. The reason is that the <html:errors> tag forced you to place HTML in the message resource file, which is not nice. This is not necessary with the new <html:messages> tag, since it sets up a loop where you may easily stick in the HTML you want. The name ActionError(s) is also misleading, since they're not only supposed to hold an error message, but, rather, any message.

Struts, however, maintains two separate queues: one for ActionErrors and one for ActionMessages. They're both held in the request object:

Collection Queue name in request object Saved by
ActionErrors Globals.ERROR_KEY =
"org.apache.struts.action.
ERROR"
saveErrors
(HttpServletRequest request, ActionErrors errors)
ActionMessages Globals.MESSAGE_KEY =
"org.apache.struts.action.
ACTION_MESSAGE"
saveMessages
(HttpServletRequest request, ActionMessages messages)

The plan is that in some coming Struts release--maybe 1.2--the ActionError(s) classes will be deprecated. My advice is therefore that you use the new ActionMessage(s) classes. There is one issue however in doing this. The Validator that was introduced in Struts 1.1 still uses the ActionErrors class. The same goes for the validate method in the ActionForm class. 

To display messages from the ActionErrors object you may use this tag:

<html:messages id="msg" message="false">
<bean:write name="msg"/><br>
</html:messages>

It's optional to specify message="false".

Earlier we saw that messages from the ActionMessages object are displayed like this:

<html:messages id="msg" message="true">
<bean:write name="msg"/><br>

Exception handling in Struts

Let me start by defining what I mean by "exception handling". It's how you handle serious errors in your program. So it's not something like forms validation errors, but more like errors that shouldn't occur with a properly set up environment. It could be a database access that fails because you've supplied an incorrect password. In Java such errors are typically handled by throwing an exception, but they could of course be handled in other ways, for example by having a method returning parameters that describe the error situation.

If a method throws an exception it's up to the receiver to determine what to do about it. It's common to re-throw the exception, but if the error can be repaired partly or completely, then the whole situation could end up in a warning message sent to the user. How messages are sent we know from the previous part of my article.

We'll first see how Struts, per default, handles exceptions and then continue by looking at how exception handling can be configured through the struts-config file. As before our point of view is the Action class, and we'll first make a small test set-up to see what throwing an exception will cause.

We'll need a jsp-page like this:

Listing 6: The tryexception.jsp page

<%@ page language="java" %> <%@ taglib uri="/WEB-INF/struts-html.tld" prefix="html" %> <%@ taglib uri="/WEB-INF/struts-logic.tld" prefix="logic" %> <%@ taglib uri="/WEB-INF/struts-bean.tld" prefix="bean" %> <logic:messagesPresent> <h3>Errors:</h3> <html:messages id="emsg"> <bean:write name="emsg"/><br> </html:messages> </logic:messagesPresent> <html:form action="tryexception.do"> <html:text property="id"/> <br> <html:submit value="Submit"/> </html:form>

In the form field we can type the name of an exception we'd like to throw from the Action class. We've prepared the page to show a list of errors if present. 

In the struts-config file we define this:

Listing 7: The struts-config.xml file

<form-beans> <form-bean name="tryexceptionform" type="org.apache.struts.action.DynaValidatorForm"> <form-property name="id" type="java.lang.String"/> </form-bean> </form-beans> <action-mappings> <action path="/tryexception" name="tryexceptionform" scope="request" input="/tryexception.jsp" type="hansen.playground.TryException"> <forward name="OK" path="/tryexception.jsp"/> <forward name="failure" path="/error.jsp"/> </action> </action-mappings>

The TryException Action class is this:

Listing 8: The TryException class

package hansen.playground; import java.io.IOException; import javax.servlet.http.*; import org.apache.struts.action.*; public final class TryException extends Action { public ActionForward execute( ActionMapping mapping, ActionForm form, HttpServletRequest request, HttpServletResponse response) throws Exception { DynaValidatorForm f = (DynaValidatorForm) form; String id = (String) f.get("id"); if (id.equals("Exception")) throw new Exception("my Exception"); if (id.equals("RuntimeException")) throw new RuntimeException("my RuntimeException"); // Back to same page return (mapping.findForward("OK")); } }

It retrieves the id of the exception to be thrown and if it's one of the ones listed it will be thrown. If not, we simply return to the jsp-page. As you can see both checked and unchecked exceptions are in the list. If we enter "Exception" in the form and submit it, we'll see something like this in the browser (I've used Tomcat 4.1.12 for the examples):

HTTP Status 500 -


type Exception report
message
 
description
The server encountered an internal error () that prevented it from fulfilling this request.
exception

javax.servlet.ServletException: my Exception
        at org.apache.struts.action.RequestProcessor.processException
                   (RequestProcessor.java:541)
. . . (traceback continues) . . .

It's the same picture if you try the other exceptions. Not very nice, is it?

Improving exception handling

Let's set up a more realistic situation to show how you can handle an exception in a better way, first without using Struts 1.1 features.

In the TryException class we'll call two methods tryChecked1 and tryChecked2, which throws two checked exceptions called MyException1 and MyException2. I don't show them here because they're simple extensions of the base Exception class. For simplicity we put the two methods in the TryException class. The first exception we convert to a message and the other one we redirect to an error page:

Listing 9: The TryException class - extended (1)

. . .
if (id.equals("tryChecked1"))
try {
tryChecked1("mydata");
} catch (MyException1 e) {
// Insert some logging code here...
ActionErrors errors = new ActionErrors();
errors.add("error", new ActionError("errors.exception"));
saveErrors(request, errors);
// Return to same page
return (new ActionForward(mapping.getInput()));
}

if (id.equals("tryChecked2"))
try {
tryChecked2("mydata");
} catch (MyException2 e) {
// Insert some logging code here...
// Save the exception for the error page
request.setAttribute("MYEXCEPTION",e);
// Return to error page
return (mapping.findForward("failure"));
}

// Back to same page
return (mapping.findForward("OK"));
}

private void tryChecked1(String s) throws MyException1 {
throw new MyException1("my Exception 1");
}

private void tryChecked2(String s) throws MyException2 {
throw new MyException2("my Exception 2");
}
. . .

First we enter "tryChecked1" in the tryexception.jsp page. The result is this:

As you can see we have defined this message in the message resource file: "errors.exception=A problem was repaired". If the exception isn't really a serious error, and we can do something sensible about it, then converting the exception to a message like this is a perfectly valid solution.

Before we try the "tryChecked2" case we'll create an error.jsp page. The first version of it simply lists all objects in request and session scope, takes the stored exception and prints it:

Listing 10: An error jsp page

<%@ page language="java" %>

<%@ page import="java.util.*" %>
<%@ page import="java.io.*" %>
<%@ page import="org.apache.struts.*" %>

<h3>In error.jsp</h3>

<%

out.println("<p><b>All Attributes in request scope:</b>");

Enumeration paramNames = request.getAttributeNames();
while (paramNames.hasMoreElements()) {
String name = (String) paramNames.nextElement();
Object values = request.getAttribute(name);
out.println("<br> " + name + ":" + values);
}

out.println("<p><b>All Attributes in session scope:</b>");

paramNames = session.getAttributeNames();
while (paramNames.hasMoreElements()) {
String name = (String) paramNames.nextElement();
Object values = session.getAttribute(name);
out.println("<br> " + name + ":" + values);
}

Object o = request.getAttribute("MYEXCEPTION");
if (o != null) {
Throwable t = (Throwable)o;
out.println("<h3>An Exception was thrown:</h3>" + t);

StringWriter sw = new StringWriter();
PrintWriter pw = new PrintWriter(sw);
t.printStackTrace(pw);
pw.flush();
String trace = sw.toString();
out.println("<pre>"+trace+"</pre>");
}

Declarative Exception Handling

Struts 1.1 offers a new way to handle exceptions: Declarative Exception Handling. Instead of coding the exception handling in the Action class, we can now specify in struts-config how a specific exception should be treated. In its simplest form we only have to enter these lines in struts-config:

Listing 12: Declaring an exception in struts config

<global-exceptions> <exception type="hansen.playground.MyException2" key ="errors.exception2" path="/error.jsp"/> </global-exceptions>

The interpretation of this is that if MyException2 is caught by Struts' ActionServlet then it should redirect to error.jsp. The key is as usual a pointer to the message resource file. This makes coding in the Action class very simple--see for yourself when we add yet another test:

if (id.equals("tryDEH")) tryChecked2("mydata");

Since the execute method declares throws Exception we don't need a try-catch block. If we enter "tryDEH" in the jsp-page we'll receive this answer:

In error.jsp

All Attributes in request scope: 
. . .  (a few lines left out here) . . .
org.apache.struts.action.EXCEPTION:hansen.playground.MyException2: my Exception 2

All Attributes in session scope: 
org.apache.struts.action.LOCALE:en

We can see that Struts has saved the exception using one of its global constants. You may use the field Globals.EXCEPTION_KEY to retrieve it from the request object, so we make a small modification to error.jsp to handle our own and now also Struts' exception:

. . . Object o = request.getAttribute("MYEXCEPTION"); if (o == null) o = request.getAttribute(Globals.EXCEPTION_KEY); if (o != null) { Throwable t = (Throwable)o; . . .

This gives us the same traceback as in Listing 11.

The exception element in the struts-config file has these attributes:

Table 1: The attributes of the exception element

Attribute Meaning
bundle Servlet context attribute for the message resources bundle
associated with this handler. The default attribute is the
value specified by the string constant declared at
Globals.MESSAGES_KEY.
[org.apache.struts.Globals.MESSAGES_KEY]
className The configuration bean for this ExceptionHandler object.
If specified, className must be a subclass of the default
configuration bean
["org.apache.struts.config.ExceptionConfig"]
handler Fully qualified Java class name for this exception handler.
["org.apache.struts.action.ExceptionHandler"]
key The key to use with this handler's message resource bundle
that will retrieve the error message template for this
exception.
path The module-relative URI to the resource that will complete
the request/response if this exception occurs.
scope The context ("request" or "session") that is used to access
the ActionError object [org.apache.struts.action.ActionError]
for this exception.
type Fully qualified Java class name of the exception type to
register with this handler.

The most interesting attribute, besides the ones we've already seen, is the handler attribute. It gives us the possibility to write a class to handle the exception. This is necessary when we, for example, will write the exception to a log file or another persistent media. It's also necessary since we want to move Java code away from error.jsp. The exception handler must extend the org.apache.struts.action.ExceptionHandler class--see the JavaDoc here. When you extend it you should override the execute method, which has this signature:

Listing 13: The signature of the execute method in an Exception Handler

public ActionForward execute(java.lang.Exception ex, ExceptionConfig ae, ActionMapping mapping, ActionForm formInstance, javax.servlet.http.HttpServletRequest request, javax.servlet.http.HttpServletResponse response) throws javax.servlet.ServletException

You get the exception as the first parameter, the data (as a bean) from the exception-element in struts-config as the second parameter, and finally the parameters you already know from the Action classes.

Here's a simple ExceptionHandler that writes the exception to the console (which is only useful during development), and forwards to the jsp-page given in struts- config.

Listing 14: An extension of an ExceptionHandler

package hansen.playground; import javax.servlet.*; import javax.servlet.http.*; import org.apache.struts.action.*; import org.apache.struts.config.*; public class MyHandler extends ExceptionHandler { public ActionForward execute( Exception ex, ExceptionConfig ae, ActionMapping mapping, ActionForm formInstance, HttpServletRequest request, HttpServletResponse response) throws ServletException { ex.printStackTrace(); return new ActionForward(ae.getPath()); } }

The entry in struts-config is this:

<exception type="hansen.playground.MyException1" key ="errors.exception1" handler="hansen.playground.MyHandler" path="/error.jsp"/>

To test it we add these lines to the Action class:

if (id.equals("tryDEH1")) tryChecked1("mydata");

If you want to code your own general ExceptionHandler you should consider adding these features:

  1. write the exception to a log (e.g. implemented through Log4J or the JDK1.4 Logging API)
  2. write as much useful data as possible to the log. If you have the userid or other user information then it could be relevant to log it so you can match the error to the user who encountered the error.
    You might have other useful information stored in session or request scope that could be written to the log.
  3. you might want to inform of the error through other channels, e.g. e-mail. Just be sure to make your set-up robust, so you don't provoke a new exception!
  4. finish by going to e.g. the error.jsp page, where you can inform the user of the error, and how he or she can proceed.

Finally I'll mention that it's also possible to add an exception handler to a specific action--e.g.:

Listing 15: Declaring a local exception handler

<action path="/tryexception" . . . <exception type="hansen.playground.MyException1" key ="errors.exception1" handler="hansen.playground.MyHandler" path="/error.jsp"/> </action>

Recap (2)

Declaring exceptions in struts-config makes coding in the Action class much simpler. Instead of having almost the same code replicated in every Action class you can now move this code to the ExceptionHandler.

Using chained exceptions

An exception typically occurs in a method in a "calling chain" initiated by the Action class. To get a realistic snapshot of the error it's very valuable if each method in this chain can add data for the logging mechanism. By using chained exceptions this is easily accomplished. Here's an example where the Action class calls a method in ClassA which calls a method in ClassB, where the exception is thrown:

Listing 16: Using chained exceptions

In the Action class: 
. . .
if (id.equals("tryDEH2"))
new ClassA().methodA("mydata");
. . .

In ClassA:
. . .
public void methodA(String s) throws MyException1 {
try {
new ClassB().methodB(s);
} catch (MyException1 e) {
throw new MyException1
("ClassA.methodA: important data . . .",e);
}
}
. . .
In Class B:
. . .
public void methodB(String s) throws MyException1 {
throw new MyException1
("ClassB.methodB: important data . . .");
}
. . .

If we simply print the Java traceback we get something like this:

hansen.playground.MyException2: ClassA.methodA: 
important data . . .
at hansen.playground.ClassA.methodA(ClassA.java:9)
at hansen.playground.TryException.execute(TryException.java:63)
. . .(several lines omitted) . . .
at java.lang.Thread.run(Thread.java:536)
Caused by: hansen.playground.MyException2: ClassB.methodB:
important data . . .
at hansen.playground.ClassB.methodB(ClassB.java:6)
at hansen.playground.ClassA.methodA(ClassA.java:7)

... 36 more

The formatting is not very nice, but the important thing is that we get the application data written out. If you're interested in more information on how to best use chained exception, and how you produce a nicer traceback, I recommend reading the article "Using Chained Exceptions in JDK 1.4".

Conclusion

Before you start coding a new Struts application it's important to decide how messages, errors and exception handling are implemented. In a large development team it's very important to have these decisions in place and documented before coding starts.  The examples and ideas presented in this article will hopefully help you simplify the decision process.

Resources




%>

When we enter "tryChecked2" we get this page back:

Listing 11: Traceback in error.jsp

In error.jsp

All Attributes in request scope: 
. . .  (a few lines left out here) . . .
MYEXCEPTION:hansen.playground.MyException2: my Exception 2 

All Attributes in session scope: 
org.apache.struts.action.LOCALE:en

An Exception was thrown:

hansen.playground.MyException2: my Exception 2 

hansen.playground.MyException2: my Exception 2
at hansen.playground.TryException.tryChecked2(TryException.java:71)
at hansen.playground.TryException.execute(TryException.java:50)
. . . (traceback continues) . . .

This page is of course only good for experimentation and testing. In a real production system you'd have to give a friendly message, format it nicely using your style sheets, and tell the user how to proceed. It's a matter of personal choice if you'll also include the raw traceback in the page. The traceback and all other useful information will have to be written to a log file or some other media, to be used by the support people. But including the traceback in the page has at least two advantages:

  1. the user may phone right away to the help desk and pass (by voice or mail) the technical information. Trained support people might recognize the error right away, and thus give immediate help to the user.
  2. if the user has seen the error before and have had some advice of what to do in the situation, she might be able to work around the problem once more.


</html:messages>

My best advice--until version 1.2 comes out and clarifies things--is this: use ActionErrors for validations, and ActionMessages for all other messages. For showing all the messages you'll unfortunately have to use both of the tags above.

Recap (1)

To have a message displayed in the browser, you first use the ActionMessage(s) classes for storing message data. You typically do this in the Action class. In the jsp-page you use the <html:message> tag to display one or more of the messages.

Install Perl module from CPAN

Using CPAN Alternatively, you can use CPAN to install the modules:

From the root prompt on your server, invoke the CPAN shell:

# perl -MCPAN -e shell

Once the Perl interpreter has loaded (and been configured), you can install modules with: install MODULENAME.

The first thing you should do is upgrade your CPAN:

cpan> install Bundle::CPAN

Once it is completed, type:

cpan> reload cpan

Now, enter the following command to retrieve all of the required modules:

cpan> install DateTime
cpan> install DBI
cpan> install DBD::mysql
cpan> install Class::Autouse

Monday, June 15, 2009

How to boot up to single mode from GRUB

At the selection menu, highlight the linux entry.
Type e
arrow down to the line which starts with kernel
Type e
go to the end of the line and space over one space
Type single and press the enter key
Type b to boot up in that mode

When you get to the prompt, you can change the root password with the passwd command.

When you are done, no need to reboot, just type init 5

Tuesday, February 10, 2009

HTML Forms



Web Programs and HTML Forms

Many Web applications require that a user provides some input that should be sent by the browser to a web server. Typically a user fills out a form by filling in some fields with text and/or selecting from lists of options. There are a number of issues to consider when designing and developing such applications, here we are focusing on just the creation of the forms themselves, not on how these forms are processed by some program running on the web server.

HTML forms provide a mechanism for including many input boxes (and buttons) that the user can use to provide input to a web application. The browser understands that when an HTML form is submitted to a web application, the values that the user input to the system must somehow accompany the request itself. The request is a normal HTTP request, there is not a separate protocol for form submission. The designer of the HTML form must include information in the form itself that tells the browser where the submission should be sent, and whether to use an HTTP GET or POST request. Although the syntax of a POST is a little different than a GET, we will just mention using GET requests here (later we will explore what happens when a browser uses a POST to submit a form).

When a user submits a form using the HTTP GET method (by pressesing a submit button, or sometimes just by pressing the Enter key), the browser will compose a HTTP GET request that includes a URI in which a "query string" is appended to the URI after a '?'. This query string will contain the names and values of all the form fields in the HTML form being submitted. The general syntax of the query string looks like this:

name1=value1&name2=value2&...&namex=valuex

Each of the names corresponds to one form field, the values are the information the user supplies (by typing in a field or selecting from lists of options).

A complete HTTP get request that includes a query string created for a form submission might look like this:

GET /cgi/addmetoyourspamlist.cgi?first=dave&last=hollinger&email=hollingd@cs.rpi.edu HTTP/1.1
User-agent: Netscape

In the above example there are 3 form fields being submitted, one named "first" that represents the user's first name, "last" is the last name and "email" is an email address. Someone had to create the actual form used, in the rest of this document we look at the HTML tags used to create forms and how the browser uses these tags to submit.

URL Encoding

You might ask: what will happen if the user types in "billy bob" as the firstname in the above example? Wouldn't this mess things up since the space character would now show up in the HTTP request line? What if the following request is sent:

GET /cgi/addmetoyourspamlist.cgi?first=billy bob&last=hollinger&email=hollingd@cs.rpi.edu HTTP/1.1
User-agent: Netscape

The server expects three words on the request line, the example above has 4!

To avoid this problem (and a few others), the browser will not send the request as shown above (with a space in the query string). Instead, the browser will encode the query string using well known encoding rules called "urlencoding". The encoded string is used as the query part of the URI. The server/web application knows about this and will decode the query string before extracting field names and values. The rules of URL encoding are shown below:

The FORM tag

All form elements must be placed between a <FORM> and </FORM>. There can be multiple forms within a single HTML document - each form is separate entity and the contents of only one form are submitted as query (although a single form many contain many elements). The FORM tag has required attributes ACTION and METHOD.

The METHOD attribute of a form tag specifies the HTTP request method that should be used when the contents of the form are submitted as part of an HTTP request. Typically the request method is either GET or POST. The web application that handles the request will receive the contents of the form and must support the method specified, or things won't work. In other words - it is possible to write a web application (CGI program or ASP script) that only handles GET requests, so when creating a form, you need to know a little about the application to decide whether to use GET or POST.

The ACTION attribute specifies where the contents of the form should be sent. This is typically a URL (could be relative or absolute), although sometimes people use a mailto: URL so that when the user submits the form it is sent to as an email message to the specified email address.

Some FORM tag examples:

<FORM method=GET ACTION=http://www.foo.com/cgi-bin/aprog>


<FORM method="GET" action="prog.cgi">

<FORM method=POST action="mailto:billy@whitehouse.gov">

Form Elements (Fields)

Between the <FORM> and </FORM> tags you define the text and fields that make up the form. You can include HTML tags inside a form to format the text however you want, and there are a number of new tags that are used to define form fields

There are a variety of types of form fields:

Input Fields

Input fields allow the user to type in a string value as input or to click on buttons or menus to select specific options. These fields are all created with the INPUT tag, the required attribute TYPE indicates what specific kind of input field is being created.

The INPUT tag also has a required attribute named NAME that establishes the name of the field being created. This name is important, since it will be sent to the Web Application along with the value the user provides. Within a single form, every input must have a unique name.

Below is a description of many of the types of input fields, see any HTML reference for a complete description

TEXT Input Fields

TEXT is the most common type of input field, it allows the user to type in a single line of text. There are some additional attributes that can control the maximum length of the field (MAXLENGTH), the size of the box drawn by the browser (SIZE) and the initial/default value for the field (VALUE).

Here are a few examples of TEXT input fields:

<INPUT TYPE=TEXT NAME=FOO>

<INPUT TYPE=TEXT
NAME=PIZZA
SIZE=10
MAXLENGTH=20
VALUE=Pepperoni>

Here is an example form that includes a couple of text fields:

<FORM METHOD=GET ACTION=cgi-bin/foo>
Your Name:
<INPUT TYPE=TEXT NAME="Name"><br>

Your Age:
<INPUT TYPE=TEXT NAME="Age"><br>

</FORM>
Your Name:
Your Age:

A couple of things to notice:

  1. There is no special kind of "number" field - we just use a text field to get the user's age (and hope they actually type in a number).
  2. The INPUTs above result only in the display of a text box, the broser does not display the name of the field (if you want the box to be labeled on the screen, you need to do that with HTML).
  3. The form above has no button to press, so there is no way for the user to submit the form!

Submission Buttons

Another type of input field is the SUBMIT type, this tells the browser to draw a button. When the user clicks on the button the browser knows to submit the contents of the form to the URL specified as the ACTION in the form tag.

Submit inputs support the attribute VALUE which is the string you want displayed in the button. If you don't include a VALUE attribute the browser will put the string "Submit" in the button. Note that the NAME attribute is not required for a submit input.

Reset Buttons

An input of type RESET tells the browser to create a button that the user can press to clear all form fields (set to the default values). You can specify the text to appear in the button with the VALUE attribute.

Now we can look at a complete form example:

<FORM METHOD=GET ACTION=cgi-bin/foo>
Your Name: <INPUT TYPE=TEXT NAME=Name><br>

Your Age: <INPUT TYPE=TEXT NAME=Age><br>

<INPUT TYPE=SUBMIT VALUE=Submit>
<INPUT TYPE=RESET VALUE="Clear Form">
</FORM>
Your Name:
Your Age:

The above form has the ACTION set to the relative URL cgi-bin/foo, which does not really exist (there is no such URI on the server). We can change the URL in the form to point to any web page or application we want, although if the URL does not reference a web application that is looking for the form fields Name and Age nothing special will happen. For example, we can specify the action as www.rpi.edu and see what happens:

<FORM METHOD=GET ACTION=cgi-bin/foo>
Your Name: <INPUT TYPE=TEXT NAME=Name><br>

Your Age: <INPUT TYPE=TEXT NAME=Age><br>

<INPUT TYPE=SUBMIT VALUE=Submit>
<INPUT TYPE=RESET VALUE="Clear Form">
</FORM>
Your Name:
Your Age:

Tables and Forms

Tables are often used to make forms look pretty - remember that you can include any HTML formatting tags you want within a form

<FORM METHOD="GET" ACTION="cgi-bin/foo">
<table><tr>
<td>Your Name: </td>
<td><INPUT TYPE="TEXT" NAME="Name"></td>
</tr><tr>
<td>Your Age:</td>
<td> <INPUT TYPE="TEXT" NAME="Age"></td>
</tr><tr>
<td><INPUT TYPE="SUBMIT" VALUE="Submit" ></td>
<td><INPUT TYPE="RESET" VALUE="Clear Form"></td>
</tr></table>
</FORM>
Your Name:
Your Age:

Checkbox Inputs

Inputs of type CHECKBOX present user with an item that can be selected or deselected. Each checkbox has a name and a value and can be initially selected/deselected. Only checkbox fields that are selected when the form is submitted are sent as part of the query. To set a checkbox to be initially checked, use the SELECTED attribute (this attribute does not nees a value.

If you don't specify a value in a checkbox tag, the browser will send the value "on". Here is an example with some checkbox definitions:

<INPUT TYPE=checkbox name=chocchip value=yes>
<INPUT TYPE=checkbox name=oreo SELECTED>

Here is a complete form example that includes some checkboxes:

<FORM METHOD=GET ACTION=cgi-bin/foo>

Select all the cookies you like:<br>

<INPUT TYPE=CHECKBOX NAME=Oreo Value=yes>
Oreo<br>
<INPUT TYPE=CHECKBOX NAME=Oatmeal >
Oatmeal<br>

<INPUT TYPE=CHECKBOX CHECKED NAME=ChocChip SELECTED VALUE=ofcourse>
Chocolate Chip<br>

<INPUT TYPE=SUBMIT VALUE=Submit>
</FORM>
Select all the cookies you like:
Oreo
Oatmeal
Chocolate Chip

You can have multiple checkboxes with the same name (each must have a different value for this to work), and the value of the field that is sent by the browser will be a collection of values for the named field (when sent to a web server, a field can have multiple values). Here is an example (you can submit this form and although there is no web application to handle it, you will be able to see the query string created by the browser).

<div BGCOLOR=yellow>
<FORM METHOD=GET ACTION=cgi-bin/foo>

Select all the cookies you like:<br>

<INPUT TYPE=CHECKBOX NAME=Cookie Value=Oreo>
Oreo<br>
<INPUT TYPE=CHECKBOX NAME=Cookie Value=Oatmeal>
Oatmeal<br>

<INPUT TYPE=CHECKBOX CHECKED NAME=Cookie VALUE=ChocChip SELECTED>
Chocolate Chip<br>

<INPUT TYPE=SUBMIT VALUE=Submit>
</FORM>
</div>
Select all the cookies you like:
Oreo
Oatmeal
Chocolate Chip

Radio Button Inputs

Radio Button inputs are like checkboxes, except that the user must select only one item from group of choices. The way the browser knows which radio buttons are part of a group of choices is by looking at the NAME attribute. All radio button inputs that form a group must have the same NAME, and each should have a different value for the VALUE attribute (this is how the web application will find out which was picked).

Here is a complete form that includes some radio buttons:

<FORM METHOD=GET ACTION=cgi-bin/foo>
Select the cookie you want to order:<br>

<INPUT TYPE=RADIO
NAME=Cookie Value=Oreo>
Oreo <br>
<INPUT TYPE=RADIO
NAME=Cookie Value=Oatmeal>
Oatmeal <br>
<INPUT TYPE=RADIO CHECKED
NAME=Cookie Value=ChocChip>
Chocolate Chip<br>

<hr>
Select the size drink you would like:<br>

<INPUT TYPE=RADIO NAME=drink VALUE=1>
<span style="font-size:small"> Small</SPAN><br>

<INPUT TYPE=RADIO NAME=drink VALUE=2>
<span style="font-size:medium"> Medium</SPAN><br>

<INPUT TYPE=RADIO NAME=drink VALUE=3>
<span style="font-size:large"> Large</SPAN><br>

<INPUT TYPE=SUBMIT VALUE=Submit>
</FORM>

Select the cookie you want to order:
Oreo
Oatmeal
Chocolate Chip

Select the size drink you would like:
Small
Medium
Large

TEXTAREA Form Fields

You can create a multiline text field with the TEXTAREA tag - you don't use the INPUT tag to create this kind of field. The TEXTAREA tag requires the NAME attribute and supports the attributes ROWS and COLS (to define the size of the box drawn on the screen). Unlike the INPUT tag - the TEXTAREA tag has an end tag, so you need to include a </TEXTAREa> tag.

Everything included between the <TEXTAREa> and </TEXTAREa> tags is the initial value of the multiline text box. The user can delete, edit or add to this initial text.

Here is a complete example:

<FORM METHOD=GET ACTION=cgi-bin/foo>

Please enter your address
in the space provided:<br>

<TEXTAREA NAME=address COLS=40 ROWS=5>
Your address goes here
</TEXTAREa><br>

<INPUT TYPE=SUBMIT VALUE=Submit>
</FORM>

Please enter your address in the space provided:

SELECT form fields

The SELECT and OPTION tags can be used to create pull-down menus and scrolling lists of choices. The SELECT tag must include a NAME attribute (this is the name of the form field sent by the browser). The SELECT tag supports the optional attribute SIZE which controls the number of options displayed at once. If the SIZE attribute is omitted or set to the value 1, the browser will display a pull-down menu. If the attribute has a value greater than 1, the browser will display a scrolling list of options.

Between the <SELECT> tag and the corresponding end tag </SELECT> there can be number of OPTION tags - one for each choice you want displayed. The text that follows the OPTION tag is the text that will be displayed by the browser. The value sent by the browser if a choice is selected can be specified with a VALUE attribute, if none is specified the value sent will be the text that follows the OPTION tag.

Below is an example that displays a pull-down menu, the OPTION values are not specified so the browser will sent the text that is displayed as each menu item:

<FORM METHOD=GET ACTION=cgi-bin/foo>

Pick your favorite:
<SELECT Name="cookie">
<OPTION>Oreo
<OPTION>Oatmeal
<OPTION>Chocolate Chip
</SELECT>
<INPUT TYPE=SUBMIT VALUE=Submit>

</FORM>

Pick your favorite:

Here is another example, this one will appear as a scrolling list (the SIZE attribute controls the size), and the OPTION values are set inside the OPTION tags:

<FORM METHOD=GET ACTION=cgi-bin/foo>
Pick your favorite:
<SELECT Name="cookie" SIZE=3>
<OPTION VALUE="1">Oreo
<OPTION VALUE="2">Oatmeal
<OPTION VALUE="3">Fig Newton
<OPTION VALUE="4">Chocolate-Chip
<OPTION VALUE="5">Chocolate Chocolate-Chip
<OPTION VALUE="6">Chippity Chocolate Chip
</SELECT>
<INPUT TYPE=SUBMIT VALUE=Submit>

</FORM>
Pick your favorite:

Form Submission

When the user presses on a submit button the following happens:

There are a few rules to keep in mind:

Checking what is sent by the browser

As many of the above forms demonstrate, you don't need to have an actual web application written to determine what the browser will send when a form is submitted. Any form the uses the GET request method includes all the form data in the URI, and this will appear in the Address box of the browser.

Other Form Field Types

There are more form field types:
Google+