CGI, or Common Gateway Interface, is one way to make Web pages dynamic and interactive. CGI is the non-proprietary protocol used by most Web servers to capture and communicate session information to and from server-side processes. In other words, it's the "glue" between the Web browser and your behind-the-scenes applications.
The purpose of this paper is to demonstrate when and how to use CGI (and its sister spec, WinCGI) on your Web site, from "hello world" to complex operations using server push and Netscape cookies - and to help you sidestep some common pitfalls in the process. It assumes you have some familiarity with programming concepts, and some degree of control over how your Web server is administered.
Many familiar applications on the Web - search engines, hit counters, guest books, feedback surveys - are CGI applications. More complex applications (such as ASU's "FASTT interactive", which accesses mainframe data through a SQL server) are also CGI applications. Many "canned" CGI applications exist, and you can write your own in virtually any programming or scripting language.
The key to CGI development is in understanding the environment the Web server creates. User input and session information are supplied to your application through environment variables and standard console input, usually in a special format that must be parsed. Your application, in turn, "speaks" to the browser through standard output, consisting of one or more HTTP headers and whatever information you choose to return.
Since CGI processes run on the server, they are browser-independent for the most part (although some special headers are not recognized by all browsers). They are also stateless; that is, each time a browser requests a given CGI application, the Web server creates a brand-new process with a brand-new environment and no recollection of any other process the server might have started or completed. The process begins with the request, and ends (generally speaking) with its output to the browser via the server.
The problem of "keeping state" among CGI applications has been addressed in a variety of ways, including passing information in URLs and hidden form fields. Another method, specified by Netscape Corporation and known as "cookies", uses special HTTP headers recognized by most (but not all) browsers. As in any programming project, the "best" method to use depends on your application.
Keywords: CGI WinCGI interactive cookies programming forms
Document updates and additional resources can be found at http://www.west.asu.edu/jenili/teachCGI/CGIpaper.html.
CGI, or Common Gateway Interface, is not a programming language. It is a protocol, a means of getting session-specific information (including user input) back and forth between the Web browser and the Web server. It's an agreement of sorts among the browser, the server, and your applications about how information will be encoded and handled.
CGI is not the only way to make Web pages that change or interact. This paper, while dealing almost exclusively with CGI, will touch on some other techniques toward the very end.
Real-world examples of CGI applications include canned search engines, hit counters, and guest books as well as custom applications such as feedback forms and student data access. The famed PizzaNet and Federal Express Tracking sites are CGI applications.
A quick note about this document.
The propeller beanie indicates examples, usually a sample with HTML
source code and program listing. I've chosen perl for the programming
language because it's a common choice for CGI and handles text strings
beautifully. Please feel free to swipe any code you can use from the
examples.
Examples open in new browser windows. When you're finished with one,
just close the window to return to this document.
Your applications must be in a directory recognized by the Web server as executable. A common first-time mistake is to create a "cgi-bin" directory in your account and assume that the files you place in that directory will be executed. They won't, unless the Web server has been told that your "cgi-bin" directory is an executable directory. This is done by the Webmaster using a server administration utility or making an entry in an initialization file.
Your Webmaster may be reluctant to give you access to an executable directory. There are very good reasons for this! An application that leaves processes unterminated can bring the Web server to its knees. An application that invokes the system to carry out a browser-entered command can leave the system exposed to threats from "hackers". On some Web servers, simply having an executable batch file on the system can allow anyone with a browser to execute any system command.
A common and reasonable administration policy is to have a central directory for CGI applications, with all applications passed to the Webmaster for review before they're made available online (if you yourself are a new Webmaster, you may want to read the WWW Security FAQ).
For some Windows-based Web servers, the application's file extension must be registered with Windows as executable (with the appropriate interpreter identified, if the application is not compiled).
On UNIX-based systems, the application must usually be world-executable. As the "owner" of the application, you can do this using the chmod command.
For scripted/interpreted language applications (e.g., perl, DOS batch, or shell), the interpreter must reside on the server. On UNIX-based systems, the interpreter must be executable to the Web server (which means world-executable, usually) and identified in the first line of the script (the #! or "shebang" line, of which you will see examples in this paper).
Compiled (e.g., C, C++, or Visual Basic) applications must be compiled for the operating system and processor the Web server resides on. For instance, if your Web server is a DEC Alpha, you must compile your application on a DEC Alpha, not on your desktop Pentium PC.
The key to CGI development is in understanding the environment the Web server creates. User input and session information are supplied to your application through environment variables and standard console input. Your application, in turn, "speaks" to the browser through standard output.
Some of this sounds complicated at times. Remember, it's nothing but input, output, and variables!
Returning output from a CGI application is as simple as "print", "write", or "echo". Your application simply sends its results through standard output, which the Web server intercepts and sends to the Web browser.
The format for this output is as follows:
| header block (one or more headers) |
Content-type: text/html |
| blank line | |
| content |
<html><body>Hello,
World</body></html> |
The header block usually consists of a MIME content-type header plus special headers as needed. The blank line is critical; without it, you'll receive a server error. The content can be plain text, formatted HTML, even the contents of a binary file if desired - as long as the content matches the type indicated by the MIME header.
Hello World: Basic output in action.
The Web server provides your application with a wealth of useful information about each session, including such goodies as the user's IP address, type of browser and operating system being used, types of images the browser will accept, and even the URL of the page from which the user linked to your site.
All of this information comes in the form of environment
variables. In perl, you can get at them using the special array
%ENV - as shown in the example below.
Show-Vars: A typical list of environment variables.
User input can be passed in a URL (pick your favorite search engine, enter something, and notice all the stuff at the end of the URL after you do) or as form input.
No matter how you get it, you're going to have to parse it. In the CGI protocol,
This sounds hairy. It's not tons of fun, but it's not too hard either. In my "GET and POST" example below, there are about a dozen lines of perl code in purple that take care of this for you. There are also "CGI libraries" on the Net that will keep you sheltered from this stuff if you wish.
Input passed in the URL after a slash (/) is available
to your application in the environment variable PATH_INFO.
Input passed in the URL after a question mark (?) is available
in the environment variable QUERY_STRING.
If you use both path info and query string, the path info must
come first. Sound like gobbledygook? Here's an example.
P's and Q's: Handling path info and query strings in a URL.
Form input is passed differently depending on the
request method used. The request method can be either GET or POST,
and is indicated in the <FORM> tag - <FORM
action="whatever" method="POST">.
If the action is omitted, the method will be GET.
Form input passed by the GET method will be in the query string mentioned above (this is why those search engines put all that stuff at the end of the URL when you enter some text).
Form input passed by the POST method is the only
thing that's not an environment variable! You'll find it
in standard input instead. However, its length will be
in the environment variable CONTENT_LENGTH.
An example with both GET and POST.
Note that the two differ by only one line.
HTTP is a stateless protocol. Each time a browser requests a given CGI application, the Web server creates a brand-new process with a brand-new environment and no recollection of any other process the server might have started or completed. The process begins with the request, and ends (generally speaking) with its output to the browser via the server.
If you want to "keep state" among CGI applications - say, form input triggers a second "intermediate" form before your application responds - you have a variety of options. Some of the more common ones are described here.
You can pass data in a URL when it doesn't need to be secure in any way, and when there isn't much of it (256 characters, to be exact). Search engines such as Lycos and Yahoo do this. The method is simple enough: Push the location of the next application, including your data in the path information or query string (see Redirecting the browser, below).
Data passed in a URL will be visible in the browser's history list. The user can bookmark a URL with data in it, which can be useful for certain applications.
If your application generates an HTML form on the
fly, you can include data in <input
type="hidden">
tags in the form. This information will then be passed to the
next application as form input, without displaying in the HTML
(although it will be visible in the source code).
Note that if your form uses the GET method, the data will be found in the query string. This is very similar to passing information in a URL.
In both methods above, the state will be lost when the user closes the browser. Netscape Corp. has specified "cookies", another technique for keeping state, in which the browser stores information in a text file on the client machine and passes that information to the specified server each time it makes a request. In this case, the browser can be made to keep state through multiple sessions, literally for years if desired. Cookies are supported by Netscape Navigator and Microsoft's Internet Explorer.
Cookies are also useful for passing relatively large amounts of data and for sensitive transactions where it is undesirable to have passed data displayed in the browser's URL history or in the source code of a document.
Cookies are set using a header with the format
Set-Cookie: NAME=VALUE; expires=Wdy, DD-Mon-YYYY HH:MM:SS
GMT; Path=PATH; Domain=DOMAIN_NAME; secure
where NAME and VALUE are, of course, the name and value of the cookie. Expires (optional) is the desired expiration date and time (GMT, in the format given), after which the cookie will no longer be sent. Domain and Path (both optional) are the server domain (which can be partial, as in .asu.edu to include www.asu.edu and www.west.asu.edu) and the path for which cookies will be sent. Secure, if included in the header, indicates that the cookie is only to be sent in a secure (SSL) transaction using the HTTPS protocol.
When the browser passes cookies to the server, the cookies will be
available to your applications in - you guessed it - an environment
variable, HTTP_COOKIE. Multiple cookies are separated
by semicolons.
Some limitations on cookies are important to note. A browser can store up to 300 cookies total, but only twenty cookies per domain. Each cookie can be up to 4,096 characters long. Also, note that the "Expires" time refers to the time on the client machine, which you can't control.
A simple cookie-setting example.
Residence-Check: A contrived example of keeping state.
The Set-Cookie header mentioned above is one of several special headers recognized by many browsers. You can send headers within the header block of your application's output, as in the CGI examples here, or as <META> tags within static HTML pages. <META> tags appear within the <HEAD>...</HEAD> block and take the form
<META HTTP-EQUIV="label"
CONTENT="value">
(where label is the part of the header that appears before the colon, and value is everything after the colon).
Some other useful headers are listed below.
You may wish the browser to request a particular page based on user input of some sort, or you may have different browser-specific versions of a page at different addresses. The syntax for this header is
Location: URL
where URL is, yes,
the URL of the page you want the browser to request. It must be
a full, not relative, URL - as in
"http://wherever.edu/whatever.html".
Where Do You Want to Go? A redirection example.
This can be useful when you want to ensure a student's
privacy in a public computing site, or perhaps when students are
taking a test online. Assuming the browser supports this header,
it ensures that the document is not saved to the hard disk
of the client machine. There are two headers that do this:
Pragma: no-cache
Expires: x
where x is the number
of seconds until the document expires (it can also be set to
now
if the browser in use is Netscape).
If your document updates regularly and frequently,
as perhaps a stock market ticker or sports scores, you may want
to update your user's view without requiring him to hit the
"Refresh"
button. The syntax for this header is
Refresh: x
where x is the time,
in seconds, that the browser will wait before pulling a fresh
copy of your page. If you want the browser to jump to a different
location after x seconds, you may specify a full URL in
the header as follows.
Refresh: x; URL
If you need to send more than one document at once
- perhaps a .ZIP file and HTML directions for unzipping and using
it - you can use the special MIME type
Content-type: multipart/mixed
followed by some unique boundary text that won't
appear in any of your documents.
Your output might look something like this:
{contents of .ZIP file, character for character}
<html>
Content-type: multipart/mixed
Here-is-my-really-really-unique-boundary-text-that-doesn't-appear-anywhere-else!
Content-type: application-compressed/zip
Here-is-my-really-really-unique-boundary-text-that-doesn't-appear-anywhere-else!
Content-type: text/html
<head>...</head>
<body>...</body>
</html>
Netscape Corp. has suggested the "Content-type:multipart/x-mixed-replace" header as a way to incorporate animation into your pages. This was at one time considered "cool", but you're probably better off using animated GIFs. They'll display more smoothly with less wear and tear on your server.
WinCGI is used on Windows-based servers with Windows-based applications (e.g., Visual Basic or Visual C++). Most Windows-based servers support both specs; the choice between CGI and WinCGI depends on the programming language you choose.
WinCGI differs from CGI in that the server writes the CGI environment to a temporary INI file and passes the name of the file to your application on the command line. Instead of reading from standard input and environment variables, your application reads from the file. The application in turn writes its output to a temporary file, which the server reads and passes to the browser.
This sounds complicated, but libraries exist in most popular languages to do all the "dirty work" for you. O'Reilly & Associates has published libraries, complete with documentation and examples, for C and Visual Basic.
As mentioned at the beginning of this paper, CGI is not the only way to make pages dynamic or interactive. Many other techniques exist, including proprietary file formats, client-side scripting and objects, server-side includes, and server-side APIs.
These techniques are not mutually exclusive; in fact, they can be used in combination to great advantage. Applications that combine techniques might include:
Your choice of technique will depend on many considerations, including the function of your application, the security required, and the degree of control you have over your audience's browser configuration. Some information on different techniques is offered in the table below.
| Client-side processing
| Server-side
processing |
|||||
|---|---|---|---|---|---|---|
| Proprietary file formats |
Client-side scripting |
Client-side objects |
Server-side includes |
CGI, WinCGI |
Server-side APIs | |
| Examples |
Plug-ins or helper apps such as Acrobat,
Shockwave |
JavaScript, Jscript, VBscript |
Java applets, ActiveX controls
| file.shtml in URL | script.cgi in form action or URL
| NSAPI, ISAPI, WSAPI |
| Type of application |
Specialized content for use in controlled
environment, or quickly bringing existing files to the Web.
| User interface control. | Varies.
| Regularly updated "flat
file". |
Action on server in response to user input or
other session-specific information.
| "Web-aware" custom
application. |
| Browser compatibility | Limited.
Helper app or plug-in for file must be installed, configured. User
operating systems may be limited.
| Limited. Browser must support
language. |
Limited. Browser must support object type, or
helper app/plug-in must be installed, configured.
| Any browser (assuming
output is HTML or text).
| ||
| Security | Depends on security
features of file; e.g., Word documents protected with a password.
| User can choose to view script. Not
good for "classified" applications.
| Objects are compiled. Could be
reverse-compiled, but are safe from the casually curious.
| User isn't given details of the
include, although the .shtml extension indicates that an include has
occurred.
| User can't view or modify
applications. Good for "classified" applications - where a
password is used to access a database, for example.
| |
| Bandwidth overhead | Depends on
file format.
| Generally low, since script is sent in
plain text.
| Can be high for first-time users, but
low afterward since objects are cached.
| Same as HTML
(low). | ||
| Server overhead | Minimal.
| Minimal. | One process per request. Can be high.
| Can be lower or higher than CGI,
depending on application.
| ||
Jeni Li Shoecraft
Support Systems Analyst, Sr.
Arizona State University West IT
4701 W Thunderbird Rd, PO Box 37100
Phoenix, AZ 85069-7100 USA
jeni.li@asu.edu
http://www.west.asu.edu/jenili
As a Support Systems Analyst Sr. in Arizona State University West's Information Technology department, Jeni's responsibilities include hardware/software testing and deployment, adaptive technology, high-level consulting with campus faculty and staff, and the usual array of "special projects". She serves as Webmaster for ASU West Web, supporting content publishers on campus and coordinating efforts with the University Web team as needed.
Jeni also works on University-level Web projects for student services. Jeni was the CGI programmer for ASU's "FASTT interactive" Web site, teamed with other university staff on its Web-based schedule of classes, and piloted editable PDF files for its "Forms Online Center".
Jeni is "ASU through and through", her B.S. in Computer Science completed at ASU in 1991 and M.Ed. in Educational Administration anticipated from ASU West in 1997.
N.A.WEB 96 - The Second International North America World Wide Web Conference http://www.unb.ca/web/wwwdev/ University of New Brunswick.