Clarion, the Web and You - Using FTP

Posted December 1 1997

Printer-friendly version

Download the code here

Welcome back! In the first two installments of this series, we examined how to use Clarion programs to work with HTTP servers on the World Wide Web. HTTP, in case it slipped your mind, stands for HyperText Transport Protocol, and it is the underlying transport mechanism for all communications between all web programs, browsers and servers. (Now, some purists may object to that, stating (quite correctly) that there are layers upon layers of protocols, but for our purposes, we don't need to go any deeper than that.)

The Internet, though, is much bigger than just the World Wide Web, and there are many other protocols which serve to connect various types of services across the net. Other examples include mail, Gopher, news and the topic of today's discussion: FTP.

About FTP

FTP stands for File Transfer Protocol, and it is designed to do just what you think it should: transfer files from one place to another across the Internet. Why should it concern us? Well, FTP is really the missing link needed to connect us and the Clarion CGI programs we have been discussing in previous installments. Uses include:

• Transferring new web pages and Clarion CGI programs from your machine to an actual web server
• Downloading data files full of information which your Clarion CGI programs have collected so that the information can be processed by in-house systems (sales requests, orders or questionnaires, for example)
• Uploading updated data files which your Clarion CGI programs use to display information on the web (catalogs, sales info, etc.)
• Sending updated data files or reports between one office and another

I'm sure once you start thinking about it, you could add quite a few more items to the above list.

We will be covering three methods of using FTP: via an interactive shareware program, via the Win 95 FTP command line interpreter, and finally, directly from within your own programs using an OOP wrapper class which interfaces with WinInet.DLL, a library of 32-bit Internet functions. Each method can be automated and run from within CW programs.

Ok, so what exactly does FTP do?

FTP provides services which you can view as being sort of a 'DOS for the Internet', such as:

• Being able to list the contents of directories
• Create, rename and delete directories
• Change from one directory to another
• Copy, rename and delete files

Before you can do any of this, though, you need to be able to connect with the machine that the files are on, and that leads us to the first step of our exploration of FTP.

At your service

In the same manner that an HTTP server handles web page requests from browsers, FTP requires that there be an FTP server at the other end of your connection to handle your file requests. In fact, what most people typically think of as a 'web server' is really a whole collection of server programs, each is designed to handle a particular kind of request. You can have machines dedicated to one type (such as a mail server or high volume HTTP server), or you can run multiple services on one machine.

What glues all of this together is still that same familiar URL (Uniform Resource Locator) which we discussed in previous installments. To review briefly:

• A URL consists of two or more parts - protocol, domain and optionally, a path within that domain to a specific file.
• The protocol indicates which server program should handle the request - HTTP for web, FTP for files, etc.
• The domain is the address of the machine you are directing the request to.

For example, 'http://www.topspeed.com' points to the web page server at www.topspeed.com, while a request of ftp://www.topspeed.com would try to connect you with the TopSpeed FTP server.

Unlike most web servers, though, which treat every request as a unique transaction, an FTP server requires that you identify yourself to it and 'login' before it will respond to any requests for files, etc. This requirement takes us to the next step...

Who are you?

Establishing an FTP session with a remote server requires logging in to that server using a userID and password. Obviously, there is no possible way for every person on the Internet to get a userID and password for every FTP server, so a convention was established whereby an FTP server can be set up to accept what is known as 'anonymous FTP'. Under this convention, you login using 'anonymous' as your userID, and your e-mail address as the password.

Note that an FTP site can be set up to be completely private (you must have a real ID and password to do anything), all anonymous or, as is the usual case, a mixture of the two. For example, a site may have some public directories that allow anonymous downloads of sales info and catalogs, and other private directories which contain information just for customers or suppliers. If you tried to access those private directories using 'anonymous', the FTP server would simply reject the login.

One other point to keep in mind is that an FTP server generally restricts your access to a small portion of the server's actual hard drive. What you see as the 'root' directory of the FTP server could be anywhere within the 'real' directory structure. Additionally, most public sites restrict uploads or don't allow them at all, for obvious security reasons. Trying to upload to a server like that will result in an 'access denied' error or something similar. If you experiment with www.topspeed.com, don't try uploading.

The task at hand

In order to keep some continuity we shall discuss how to accomplish the same tasks using each system. These tasks will be to a) login to the server, b) retrieve a directory listing, c) change to a sub-directory, d) download two files, e) upload one file and, finally, f) terminate the session. We'll call our fictitious server 'www.sample.com', but you could easily try many of the same commands using the anonymous FTP server at www.topspeed.com. You could even download the latest C4 beta.<G>

A quick note about testing as well: the MS Personal Web Server (PWS, also sometimes called the Peer Web Server) does also include an FTP server, so you can use it for experimenting with any of the following methods without needing to actually connect to the Internet. By default, it uses a domain name of 'localhost'.

Let's begin...

Using the Win95 FTP Command line processor

The Windows 95 FTP command line processor is a TCP/IP utility that is installed when TCP/IP networking is installed. Instructions on its use are not installed by default - you need to install the network and administration help options from your Win95 CD. Once you have those help files, look under 'Appendixes' > 'Command-Line Commands Summary' > 'TCP/IP Utilities' > 'FTP' for instructions (hey, I didn't say Microsoft made them easy to find!).

This utility is the personification of what I said earlier about FTP being like DOS. To run it, you press START, then RUN and then type in 'ftp'. A pseudo-DOS session opens up, with a command prompt of 'ftp>'. That's it. If you don't know what to do next, you are out of luck (wow, it IS like DOS!). All interaction with FTP servers is done via commands typed in on the command line. For example, type 'help' and press enter to see a list of supported commands (that's it - no explanations, just the list). To accomplish our tasks, we would type the following, pressing enter after each command:

open www.sample.com
anonymous
me@somewhere.net
dir
cd public
binary
get file1.tps c:\download\file1.tps
get file2.tps c:\download\file2.tps
put c:\upload\local.tps local.tps
disconnect
quit

Of course, after each command we would see things happen, such as being prompted for our userID ('anonymous'), see a file listing, etc. To explain a few of the less obvious commands: OPEN - establishes a connection and logs into the FTP server, BINARY - sets the mode for the file transfers, DISCONNECT - breaks the server connection, QUIT - ends the FTP program.

When transferring files, the first name specifies the original file name and location while the second specifies the new filename and location. For example, we used GET to download file1.tps from the current directory on the FTP server ('public/.html') and place it in our local c:\download directory.

The actual session would look something like this:

FTP at the DOS prompt - At least it works

An portion of an interactive session using the Win95 FTP command line utility

To automate the session from within a CW program, you would put the above commands into a text file (we'll call it script.txt) and then use the Clarion RUN() function like this:

Note that the userID of 'anonymous' is automatically sent in response to the 'User' prompt, and the e-mail address in response to the password prompt. See the Help file documentation (skimpy as it is) for a complete list of the command line switches and FTP commands.

Why use this utility? Well, you can't beat the price and it does work.

Using CuteFTP

CuteFTP is an award winning shareware FTP utility, which is very powerful and easy to use. When run interactively, it presents session information, commands and file listings in various windows, allowing point and click file operations, drag and drop file transfers and more. A portion of our example session looks like this (right after logging in):

FTP the Windows way

Note that we didn't have to type any of the commands - CuteFTP is simply showing us what is going on. The bottom left file window contains the contents of a local directory, while the right one contains the contents of the root directory of the www.sample.com FTP server. Note the 'public' directory where we would place our files.

CuteFTP also provides basic scripting functions. Commands like 'Local=' and 'Remote=' set the active directory for the local and remote machines, while 'Get=' and 'Put=' transfer files to and from those directories. A script (actually just a text file) to perform our tasks would look something like this:

Host=www.sample.com
anonym
Retry=3
Local=C:\Download
Get=/public/file1.tps
Get=/public/file2.tps
Remote=/public
Local=C:\Upload
Put=local.tps

To automate the session from within a CW program, you would run CuteFTP using a command like this:

RUN('C:\CuteFTP\CUTFTP32.EXE script=c:\temp\cuteftp.txt')

If you need a general purpose FTP utility, CuteFTP is a really nice program. A newer 2.0 version includes even more scripting and macro capabilities than the one shown here, and can be downloaded from www.cuteftp.com. Registration is a mere $34.95 US.

Using the Microsoft Win32 Internet functions (WININET.DLL)

The Microsoft Win32 Internet functions are a high level library provided by Microsoft which encapsulate most of the common functions needed to Internet enable an application. Basic functions are provided to establish the Internet connection, and then you can use other HTTP, Gopher and FTP specific functions to connect with servers running those services and interact with them using that connection.

On the plus side, these functions really do insulate you from the down and dirty details of TCP/IP, Sockets and other arcana which are part of programming low-level internet applications. On the minus side, they only get you three quarters of the way there - at least from the CW perspective. Why? Read on...

Doing the prototype two step

Since these are Win32 API calls, using these functions requires a good knowledge of how to prototype API functions and data structures in Clarion. This process is easier than it used to be since equates are now provided with CW which help map the common C/C++ data types to their corresponding Clarion equivalents.

Why do you even have to care about C++ prototypes? Unfortunately, almost all Microsoft documentation assumes you must be using C++ and hence all examples are shown using C++ syntax. This means you would need to translate those prototypes into Clarion compatible code before you can call the functions.

For example, the API prototype for the basic function 'InternetOpen' looks like this in the MSDN documentation:

HINTERNET InternetOpen(
IN LPCTSTR lpszAgent,
IN DWORD dwAccessType,
IN LPCTSTR lpszProxyName OPTIONAL,
IN LPCSTR lpszProxyBypass OPTIONAL,
IN DWORD dwFlags
);

Doesn't look very familiar, does it? Fortunately, CW now has an include file called 'WinEqu.clw' which equates most of these C++ data types for you. A relevant excerpt looks like this:

WORD EQUATE(USHORT)
DWORD EQUATE(ULONG)
HANDLE EQUATE(UNSIGNED)
LPCSTR EQUATE(CSTRING)
HWND EQUATE(HANDLE)
HINSTANCE EQUATE(HANDLE)

Note how DWORD, for instance, maps to a Clarion Ulong.

The InternetOpen function, then, can be prototyped in CW as:

InternetOpen(*lpcstr,dword,dword,dword,dword), unsigned,pascal,raw,NAME('InternetOpenA'),dll(dll_mode)

Still pretty ugly, eh? To top it off, there are pages of other values which are used to set flags, interpret return codes, etc. when calling the WinInet functions, not to mention the fact that some of them just return pointers to memory buffers which then need to be copied into local strings before they can be accessed - all the kinds of things which Clarion usually handles for us so nicely.

In any case, InternetOpen() simply sets up all of the internal structures, etc., that Windows needs to begin an Internet session. So, assuming you have all of the other equates and structures defined, a simplified call to InternetOpen() would look like this:

!-- Agent passes the name of the application (Cstring)
lpzAgent = 'WEBSYNC'
!-- Returns a handle for the Internet session
hSession = |
InternetOpen(lpzAgent,INTERNET_SERVICE_FTP,ZERO,ZERO,ZERO)
!-- If call fails, the handle is zero
If hSession = 0
Message('InternetOpen() Failed|Code: ' & |
GetLastError())
End

Note how, if the function failed, we called another API function named 'GetLastError'. In this case, the function might return any of over 40 Internet related error codes.

If our call to InternetOpen() did succeed, the next step would be to actually make a connection with an FTP Server. To do that, we would call InternetConnect(). The prototype for this function looks like this:

InternetConnect(unsigned,*lpcstr,word,dword,dword,dword,|
  dword,dword),unsigned,pascal,raw,name('InternetConnectA'),|
 dll(dll_mode)

and the code to use it would look like this:

!-- Server name (Cstring)
lpzServer = 'www.sample.com'
!-- Use anonymous FTP (both Cstring)
lpzUser = 'anonymous'
lpzPassword = 'me@somwhere.net'
!-- Attempt to connect to the server
!-- Parms in all uppercase are WinInet equates
dwContext = eFTPConnect
hConnection = InternetConnect( |
hSession,                      |! Handle from InternetOpen()
lpzServer,                     |! FTP server to connect to
INTERNET_DEFAULT_FTP_PORT,     |! FTP port (21)
lpzUser,                       |! User Name for FTP login
lpzPassword,                   |! Password for FTP login
INTERNET_SERVICE_FTP,          |! Connection type
zero,                          |! Flags (active/passive)
dwContext)                      ! Callback context
!-- Test for success (0 = no handle/failed)
IF NOT hConnection
  MESSAGE('InternetConnect() Failed|Code: ' & |
  GetLastError())
END

At this point if all went OK we are now logged into the root directory of the www.sample.com FTP server. To download our files from the public directory, we will now use two FTP specific WinInet calls - FtpSetCurrentDirectory and FtpGetFile. Note that we could avoid calling FtpSetCurrentDirectory by passing the full pathname of the file to download, but that would be making it too easy, right?

!-- Set directory we want to change to (Cstring)
lpzCurrentDirectory = 'public/.html'
!-- Make the change (uses the handle from InternetConnect())
Result = FtpSetCurrentDirectory(hConnection, |
lpzCurrentDirectory)
If Result = False
Message('Change Directory Failed|Code: ' & |
GetLastError())
End
!-- Setup the remote and local filenames (Cstrings)
lpzHostName = 'File1.tps'
lpzLocalName = 'C:\Download\File1.tps'
!-- Now download the file
Result = FtpGetFile(      |
hConnection,              |! Handle from InternetConnect()
lpzHostName,              |! Name of file on FTP server
lpzLocalName,             |! Local name to save file
False,                    |! Stop if local file exists
Zero,                     |! File attributes for new file
INTERNET_FLAG_DONT_CACHE +
FTP_TRANSFER_TYPE_BINARY, |! Download flags
dwContext)                 ! Used by callback functions
IF Result = False
  MESSAGE('Download File1 Failed|Code: ' & |
  GetLastError())
END

Downloading the second of our two examples would work exactly the same way. Our last task, uploading a file, is quite similar only it uses FtpPutFile():

!-- Setup the remote and local filenames (Cstrings)
lpzHostName = 'local.tps'
lpzLocalName = 'C:\Upload\local.tps'
!-- Upload the file to the current FTP directory
Result = FtpPutFile( |
hConnection, |! Handle from InternetConnect()
lpzLocalName, !| Local file to upload
lpzHostName, !| Name of file on FTP server
FTP_TRANSFER_TYPE_BINARY, !| Transfer type
dwContext) ! Used by callback functions
!-- do typical error checking

All right, we've done what we needed to do. Now it's time to clean up and tell Windows that we don't need the resources used by our Internet connections any more. This is accomplished by using one last function called InternetCloseHandle(). This function will close an Internet handle and any handles below it. In our case we currently have a two level hierarchy of handles - hSession contains our overall session handle and hConnect contains the connection handle to our FTP server within our session (remember, we passed the session handle to InternetConnect()). So, we could close just the FTP connection by calling InternetCloseHandle and passing it hConnect, leaving hSession still active so that we could, for example, open another connection with a different FTP server.

For our purposes, though, we will pass it the overall session handle and let it automatically close the FTP connection for us. This would look like this:

!-- Release all handles within the session
Result = InternetCloseHandle(hSession)
IF Result = False
  MESSAGE('Close Handle Failed|Code: ' & |
  GetLastError())
END

Whew! That wraps up our little excursion into using a few of the Win32 Internet functions. Some of the other functions we didn't cover here include ones for creating, renaming and deleting files and/or directories and others for retrieving directory listings (sort of the Internet equivalents of the Clarion Directory() function).

If you'd like to pursue using WinInet, here's what to do:

• Download the file that accompanies this article, then follow the instructions inside it. It contains all of the basic WinInet functions already prototyped for Clarion, as well as all of the basic equates, flags and structures used by the functions. Also included is a lib file for the WinInet.DLL.
• Beg, borrow or steal (just kidding!) a Microsoft MSDN CD (Microsoft Developers Network), and print out the sections which describe the WinInet functions (about 60 pages). I believe there are also books that describe how to use the Win32 Internet functions, but I have always used MSDN myself.
• Remember that your applications must be compiled as 32-bit.

Isn't there a better way to do this?

Can the WinInet/API beast be tamed? Sure!

The simplest way to do it is with CW 'wrapper' functions. These functions would act as a mediator, abstracting us one more step up the conceptual ladder, and allowing us to hide the more complex API calls beneath a much friendlier native CW interface.

By strange coincidence, I am in the process of doing just that - wrapping WinInet up in Clarion OOP classes. Using the classes, the example process we have been following would look more like this snippet of code:

!-- Open a connection
FTP.Open('WebSync')
IF FTP.ErrorCode THEN FTP.DisplayError.
!-- Connect with the server anonymously)
FTP.ConnectToServer('www.sample.com')
IF FTP.ErrorCode THEN FTP.DisplayError.
!-- Download a File
FTP.DownloadFile('public/file1.tps','c:\download\')
IF FTP.ErrorCode THEN FTP.DisplayError.
!-- Close the session
FTP.Close
IF FTP.ErrorCode THEN FTP.DisplayError.

Bit easier to deal with, in my mind. If there is sufficient interest, I may continue this series at some point in the future with another article describing the classes and the application they are being used in (a point and click, drag and drop FTP utility written entirely in CW). Send me feedback if you would like to see that article.

Summary

FTP is a tremendously useful tool, and I hope I have given you some ideas on how you can incorporate it into your own programs.

So try it... you'll like it!

Article comments