Cryptography in .NET
What is Cryptography?
Cryptography is all about converting plain text or clear text into cipher text through a process known as encryption. The cipher text is converted back to plain text or clear text by the opposite process called decryption:
The mathematical cryptography algorithm that performs the encryption and decryption transformations is also called a cipher and the encrypted text is called the cipher text.
Let's look at a simple example. Let's build a simple ASP.NET page.
All this ASPX page does is to display 'Some Information!' in a label server control in the Page_Load() event. Pretty simple, isn't it? Let's see what the page looks like in IE 6:
Let's use a tool called TCPTrace.exe, which is a network-sniffing tool. Microsoft also ships a similar tool called Network Monitor with the Windows Server products.
Important The TCPTrace.EXE tool can be downloaded from www.PocketSOAP.com
My web server (IIS 5.0) is running on port 80 and in order to sniff the packets sent to my web server, we've to change the port on the TCPTrace utility to 81 and the TCPTrace utility will forward all the requests received at port 81 to port 80 (to the web server). When accessing the site from IE use port 81. For example, if you're accessing your localhost then you've to access it like this: http://localhost:81/
As you can see, anyone who has access to a simple network-sniffing tool can read the information that we've transmitted over the public network. So how do you prevent this information leak? Cryptography plays a major
Why Use Cryptography?
If your computer is connected to or transmits information over an electronic network your data is visible to everyone and is available for hackers. Today, more and more companies are doing business online and this increases the security risk for the companies online business as well as the customers and partners that interact with the company. To address these problems, each and every company must take strong steps to protect their online business as well as their customers and partners.
When used properly, cryptography addresses the following problems:
Confidentiality- confidentiality assures that your information is protected.
Authentication- authentication assures that you know who is accessing your private network.
Integrity- integrity assures that information is not being tampered with during transit.
Non-repudiation- non-repudiation assures that the sender can't deny sending the message.
Cryptography provides all the services to address the security and privacy concerns of transmitting sensitive data over a public network.
Concepts of Cryptography
A cryptography algorithm is a mathematical function that transforms the readable plain text message into unreadable text garbage, and reverses the process to produce readable text from an encrypted message. All cryptographic algorithms are based on two simple principles:
Substitution- the concept of substitution is based on simply replacing every character in the message with another one. For example, when using the Caesar cipher, for a given letter in the message, shift to the right (in the alphabet) by three. That is, an 'a' becomes 'd', 'b' becomes 'e', and so on. This can be generalized to work for any number n not greater than 25 (assuming a 26 letter alphabet). In this cipher, the number n is the 'key'.
There are many varieties of the substitution ciphers available, including mono-alphabetic substitution ciphers, poly-alphabetic substitution ciphers, and perfect substitution ciphers.
Transposition- the concept of transposition is based on scrambling the characters that are in the message. Some common forms of transposition algorithm involve writing the message into a table row-by-row and reading them column-by-column. Some of the transposition algorithms such as Triple-DES perform this process three times to create the cipher text.
The math formula of cryptography is simple. When you pass the plain text message into the encryption function, the function should provide the CipherMessage, and when you pass the CipherMessage into the decryption function, it should return the original message:
Encryption(Message) = CipherMessage
And:
Decryption(CipherMessage) = Message
In the same way, the following formula should also be true:
Decryption(Encryption(Message)) = Message
In the simple encryption/decryption algorithms, the algorithm is well known, and anyone who knows the algorithm should be able to decrypt the CipherMessage. Therefore, to provide more security for the algorithms, a key is added.
We need a key to lock and unlock a lock. In the same way, a cipher is a math algorithm and a key is a sequence of bytes that is used to encrypt and decrypt the information. If we don't have the key, we can't unlock the lock. In the same way, if we lose the key we can't decrypt the encrypted data. The keys come in different sizes based on the cryptography algorithms. For example, the DES algorithm is based on 56 bits and the RC2 algorithm is based on 128 bits.
In order to encrypt or decrypt the message we have to pass the appropriate key into the function:
Encryption(Message, Key) = CipherMessage
And:
Decryption(CipherMessage, Key) = Message
In the same way, the following formula should also be true:
Decryption(Encryption(Message, Key), Key) = Message
Cryptographic Algorithms
Cryptographic algorithms can be divided into three types:
Symmetric algorithms- in symmetric cryptographic algorithms, the same key is used for encrypting and decrypting the message.
Asymmetric algorithms- in asymmetric cryptography algorithms, different keys are used for encrypting and decrypting the messages. The asymmetric cryptography algorithms are also known as public key infrastructure (or PKI).
Hashing or Message Digest algorithms- in hash cryptographic algorithms, the original text is transformed into a fixed-length cipher text. The hash algorithms also perform one-way encryption meaning the hashed cipher text can't be decrypted to its original clear text version. The fixed length of the cipher text changes based on the algorithm from 128 to 256 bits.
Symmetric Algorithms
When encrypting and decrypting the information, the same key is used for the encrypting and decrypting process in symmetric algorithms, as shown in the following figure:
With this method, the data can be transmitted in an insecure network and the receiver can decrypt the information using the same cryptography algorithm used by the sender. Of course, the key that is used to encrypt and decrypt should be kept secret if you want this method to work for you. Another problem with this approach is distributing the key to the other end where it needs to be encrypted or decrypted.
For more information about the key sizes, read the paper 'Selecting Cryptographic Key Sizes' at www.Cryptosavvy.com/Cryptosizes.pdf.
Some of the common symmetric encryption algorithms are:
DES- the Data Encryption Standard was adopted by the US Government in 1977, and by ANSI in 1981. DES follows 56-bit key for encryption and decryption. DES is a very famous algorithm but due to its small key length support, its use is very limited in today's world.
TripIe-DES- Triple-DES (or 3DES) is a very secure algorithm when compared with DES, since Triple-DES encrypts the message three times using the DES algorithm with different keys. The total key length of Triple-DES is 168 bits.
Blowfish- Blowfish is a fast, compact, and simple encryption algorithm invented by the famous author Bruce Schneier, who is the author of the celebrated book 'Applied Cryptography' (John Wiley & Sons, ISBN 0471128457). This algorithm allows a variable key length up to 448 bits.
IDEA- the International Data Encryption Algorithm (IDEA) was developed by James L. Massey and Xuejia Lai in Switzerland. Widespread use of this algorithm was hindered by several patent problems.
RC2, RC4, RC5- the RC2 and RC4 algorithms were originally developed by Ronald Rivest for RSA Security. Both RC2 and RC4 allow key lengths between 1 and 2,048 bits. On the other hand, RC5 allows a user-defined key length, data block size, and number of encryption rounds.
Rijndael (AES)- this algorithm was originally developed by Joan Daemen and Vincent Rijmen. Rijndael is a fast and compact algorithm that supports keys of 128, 192, and 256 bits in length.
Important The .NET Framework supports DES, Triple-DES, RC2 and Rijndael symmetric encryption algorithms.
Symmetric key algorithms are much faster than PKI algorithms, so they are the preferred choice for encrypting and decrypting large blocks of data. Also, they're also very easy to implement. The main disadvantage of symmetric encryption is that we need to protect the keys, and it is a challenge to exchange the keys between the encryption source and the decryption destination. The security of the algorithms is also related to the length of the key: the longer the key, the slimmer the probability of the information being decrypted. The possibility of information being decrypted is also based on the complexity of the key that you've chosen. The more complex the key, the slimmer the probability of the information being decrypted.
Asymmetric Algorithms
With asymmetric algorithms, one key encrypts the message and the other key decrypts the information as shown in the following figure:
Both the keys are different but they're related to each other. Therefore, we can publish the public key without worrying about the possibility of our encryption being compromised. This encryption is also called public-key encryption, or Public Key Infrastructure (PKI), since the public key is available publicly without compromising the integrity and security of the key or message. The decryption key is normally called the private key or secret key.
Public-key cryptography and related standards and techniques underlie security features of many products, including signed and encrypted e-mail, form signing, object signing, single sign-on, and the most popular Secure Sockets Layer (SSL)/TLS protocol.
As shown in the figure above, the public key can be freely distributed, but only you will be able to read a message encrypted using this key. When someone sends an encrypted message to you, they encrypt the message with your public key, and upon receiving the encrypted data you can decrypt it with the corresponding private key. The message encrypted with the public key can be decrypted only with the corresponding private key.
Important The PKI systems also use key exchange methods such as 'Diffie-Hellman key exchange'. The Diffie-Hellman key exchange is not an algorithm-it is a method that allows us to develop secure key exchange between two parties. The other methods are Digital Signature Standard (or DSS) and Elliptic Curve Cryptosystems.
The other way is to download the public key from an online repository. For example, if you get a client certificate from Thawte, you've an option to add your public key to the online repository. When someone wants to send something very secure all they've to do is to get the public key from you or from the online repository and sign the information with your public key and send it to you. When you receive the message, you decrypt it with your private key. This is one of the most common ways of exchanging the key.
Compared with symmetric-key encryption, public-key encryption requires more computation and is therefore not always appropriate for large amounts of data.
The .NET Framework supports two asymmetric algorithms:
DSA/DSS- Digital Signature Standard (DSS) was developed by the National Security Agency. DSS is based on the Digital Signature Algorithm (DSA), and it supports any key length.
RSA- RSA is a well-known public key algorithm developed by Ronald Rivest, Adi Shamir, and Leonard Adleman, and supports variable key length based on the implementation.
Message Digest Algorithms
Message Digest Algorithms (also known as MAC or hash algorithms) transform a variable-size input and return a fixed-size string as shown in the following figure:
The hash algorithms are also called one-way hash, since the hashed string can't be converted back to the original state-once the original clear text value has been hashed, it is not possible to get the original hashed value from the MAC.
When we hash a clear text message with a hash algorithm, it uses a key to produce the hash value. Once the hash value is generated, we can send the clear text and the hash to the other end, where the clear text value can be hashed with the same key and algorithm-the generated hash value can then be compared with the one supplied by the other end. If the values match, we can be sure that the message has not been altered in transit.
Important ASP.NET's Forms authentication module supports the one-way hash algorithms MD5 or SHA3 to authenticate the usernames/passwords stored in the config.web file. The same functionality can also be extended by storing the hashed passwords in the database using the HashPasswordForStoringInConfigFile method of the FormsAuthentication class available in the System.Web.Security namespace. For more information, have a look at 'Professional ASP.NET Security' (Wrox Press, ISBN: 1-86100-620-9).
Some of the common MAC functions are MD2, MD4, MD5, SHA, SHA-1, SHA-256, SHA-384, and SHA-512. The .NET Framework supports the following hash functions:
HMACSHA-1
MACTripleDES
MD-5
SHA-1
SHA-256
SHA-384
SHA-512
A Hash Message Authentication Code (or HMAC) function is a technique for verifying the integrity of a message transmitted between two parties that agree on a shared key.
Digital Signatures
Although encryption and decryption address a few problems, there are two important problems they don't address:
Tampering
Impersonation
Digital signatures use the one-way hashing functions for tamper detection and related authentication problems. Since the value of the hash is unique for the hashed data, any change in the data, even deleting or altering a single character, results in a different value. Moreover, the content of the hashed data cannot, for all practical purposes, be deduced from the hash. This therefore becomes the best way to detect tampering.
In public key encryption, it's possible to use the private key for encryption and the public key for decryption. Since this could create problems when encrypting sensitive information, we can digitally sign any data, instead of encrypting the data itself. Signing the data creates a one-way hash of the data, which can then be encrypted using the private key. The encrypted hash, along with other information, such as the hashing algorithm, is known as a digital signature. The following figure shows a simplified view of the way a digital signature can be used to validate the integrity of signed data:
If you look at the above figure, the sender sends the clear text (or encrypted) message with the Digital Signature. The Digital Signature is computed based on the clear text message and the clear text message is hashed using a hashing algorithm such as MD5 and the hashed value will be signed by the private key. At the other end, we'll receive the clear text message (or encrypted) with the digital signature. Then we'll compute a hash value for the clear text and compare it with the Digital Signature. If the Digital Signature verification process was successful then, we're assured that the data has not been tampered with in transit.
Cryptography Terminology
Before diving into the world of crypto coding, we need to understand some cryptographic terminology.
Block Ciphers and Stream Ciphers
Cryptographic ciphers handle data in two formats:
Block ciphers
Stream ciphers
Block ciphers are traditionally the most popular ones. A block cipher transforms a fixed-length block of plain text data into a block of cipher text data of the same length and then repeats the process until the entire message has been processed. This transformation takes place under the action of a user-provided secret key. Decryption is performed by applying the reverse transformation to the cipher text block using the same secret key. The fixed length is called the block size, and for many block ciphers, the block size is 64 bits. Typically, symmetric algorithms are based on the block cipher format. For example, the DES and RC2 algorithms use 8 bytes, 3DES uses 16 bytes, and Rijndael uses 32 bytes as input. Using this scale each algorithm splits the input into the blocks and performs the transformation.
More recent symmetric encryption algorithms are based on stream ciphers. Every stream cipher generates a keystream and encryption is provided by combining the keystream with the plain text (usually with the bitwise XOR operator). Stream ciphers can be designed to be exceptionally fast, much faster in fact than any block cipher. While block ciphers operate on large blocks of data, stream ciphers typically operate on smaller units of plain text, usually bits. The encryption of any particular plain text with a block cipher will result in the same cipher text when the same key is used. With a stream cipher, the transformation of these smaller plain text units will vary, depending on when they are encountered during the encryption process.
Padding
Block ciphers deal with blocks of bits (usually 64 bits), and the last remaining bits may not fit in a block. For example, suppose we have 136 bits of information that we are trying to encrypt using a block cipher that takes 64 bits (or 8 bytes) at a time to encrypt. The following figure shows how the 136-bits input is split into 64-bit blocks for the padding process.
In the process of encryption, the first two blocks will contain 128 bits, and the remaining eight bits will not fit in the block cipher's buffer. To address the incomplete block, padding is needed. A padding scheme will define how the last incomplete block of data will be handled in the process of encryption. The padding will be addressed in the process of decryption by removing all the padded characters and restoring the original text.
PKCS#5 (or Public Key Cryptography Standard) is one of the most famous padding schemas and it was published by RSA Security, Inc. For more information, please visit the RSA web site at http://www.rsa.com/rsalabs/pubs/PKCS/.
The .NET Framework handles padding using the PaddingMode enumeration. The PaddingMode enumeration supports three values:
None
PKCS7
Zeros
As the name suggests, when we use None, no padding is done. When PKCS7 is used, the remaining number of blocks will be filled with the remaining number of bits. For example, if six bits are free in a given byte, the last six bits will be padded with the value six. If four bits are free, the last four bits will be padded with the value four:
When PaddingMode.Zeros is used, the remaining bytes will be filled with the value zero:
Modes
The mode of a cipher determines how blocks of plain text will be encrypted into blocks of cipher text, and decrypted back. The CipherMode enumeration defines the block cipher mode to be used when performing the encryption or decryption process. You can specify the mode using the Mode property of many of the cipher algorithms. The .NET Framework supports CBC, CFB, CTS, ECB, and OFB modes.
When using ECB (or Electronic Code Book) mode, each block of plain text is encrypted to a block of cipher text. The main drawback to ECB mode is that the same plain text will always encrypt to the same cipher text when the same key is used.
The CBC (or Cipher Block Chaining) mode overcomes the drawbacks of ECB mode. When using CBC mode, each block of plain text is combined with the previous block's cipher text (using an XOR operation), which produces encrypted blocks of cipher text:
Since there is no previous cipher text block when starting the process, an initialization vector (or IV) is used for the first block of plain text.
The CFB (or Cipher Feedback) mode allows a block cipher to act like a stream cipher by processing the small increment of plain text into cipher text instead of processing it block by block. CFB mode also uses an IV to process the initial plain text.
The CTS (or Cipher Text Stealing) mode is a very versatile mode, which behaves pretty much like CBC mode. The CTS mode handles any length of plain text and produces cipher text that matches the length of the plain text.
The OFB (or Output Feedback) mode works pretty much like CFB mode; the only difference is the way that the internal buffer (shift register) is handled.
The System.Security.Cryptography Namespace
The System.Security.Cryptography namespace provides a simple way to implement security in your .NET application using the cryptography classes. Some of the cryptography classes are pure .NET managed code, and some of them are wrappers for the unmanaged Microsoft Crypto API. You can find out if the cryptography class is a managed or unmanaged code by looking at the class name. All the unmanaged providers end with the suffix CryptoServiceProvider, and all managed providers end with the Managed suffix. For example, if you look at the hashing classes such as MD5CryptoServiceProvider, SHA1Managed, and SHA256Managed, you can figure out that SHA1Managed and SHA256Managed are pure .NET managed implementations of the SHA algorithm.
The Crypto API is Microsoft's API for accessing cryptographic functions built in to the Windows platform. Microsoft recently released CAPICOM, an ActiveX wrapper around the Crypto API to simplify Crypto API programming in Visual Basic 6, but it implements only a subset of the API.
Note that the .NET Framework supports the use of strong key lengths in all encryption algorithms. However, for encryption algorithms that are implemented on top of Crypto API, you need to install a High Encryption Pack to upgrade your version of Windows:
For Windows 2000 users, Service Pack 2 includes the High Encryption Pack. It can also be obtained from the following URL: http://www.microsoft.com/windows2000/downloads/recommended/encryption/.
For Windows NT 4.0 users, Service Pack 6a includes the High Encryption Pack; this can be downloaded from: http://www.microsoft.com/ntserver/nts/downloads/recommended/SP6/allSP6.asp.
For Windows ME, Windows 98, and Windows 95 users, Internet Explorer 5.5 includes the High Encryption Pack, or you can download it from: http://www.microsoft.com/windows/ie/download/128bit/default.asp.
Cryptography Class Hierarchy
The System.Security.Cryptography namespace provides three top-level classes, SymmetricAlgorithm, AsymmetricAlgorithm, and HashAlgorithm, representing the three main areas of cryptography as shown below:
This model also brings the flexibility of extending the namespace. For example, the System.Security.Cryptography namespace doesn't currently support the Blowfish symmetric algorithm. If we wanted to add this algorithm to this namespace, all we would have to do is to derive our Blowfish algorithm class from the SymmetricAlgorithm class, and we'd get most of the standard functionality for free. The next advantage is that all the algorithm provider classes are inherited from their algorithm implementation classes. For example, the SHA1 hashing algorithm provider (the SHA1Managed class) is derived from the SHA1 hash algorithms implementation (the SHA1 class).
Hashing with .NET
The System.Security.Cryptography namespace in the .NET Framework provides several interfaces, known as Cryptographic Service Providers (CSPs), which implement a variety of hashing algorithms and make hashing simple and straightforward.
As we saw earlier, the .NET Framework implements several well-known, secure hash algorithms, including Message Digest 5 (or MD5) and Secure Hash Algorithm (or SHA). The MD5 provider generates 128-bit hash values, and the SHA provider can generate 160-bit, 256-bit, 384-bit, and 512-bit hash values. The ComputeHash() method of both the MD5 and SHA CSPs accepts a byte array or a Stream object, and returns a hash value.
The .NET Framework also implements a key-based hash, which is often used to generate digital signatures.
The HashAlgorithm Class
All the hash algorithm classes are inherited from the HashAlgorithm abstract class. The HashAlgorithm class exposes some common methods and properties that can be used across all the hashing algorithms. The following table discusses a few of them:
Class Member
Description
Hash
Returns the computed hash value in a byte array. Hash is a public property and HashValue is a protected field-both return a byte array.
HashValue
HashSize
Returns the size of the hash in bits. HashSize is a public property and HashSizeValue is a protected field-both return an integer value.
HashSizeValue
InputBlockSize
A public property that returns an integer representing the input block size in bits.
OutputBlockSize
A public property that returns an integer representing the output block size in bits.
ComputeHash()
Computes the hash value for the given input byte array or Stream. ComputeHash() is a public method that returns the output in a byte array or Stream object.
Create()
Creates an instance of the hash algorithm that is currently in use. For example, if you are using an MD5 hash algorithm then it'll create an object type of that algorithm. Create() is a static method.
TransformBlock()
Generates the hash value for a given range of the input byte array and copies the result into another byte array. TransformBlock() is a public method that returns a byte array.
TransformFinalBlock()
Generates the hash value for a given range and returns a byte array. TransformFinalBlock() is a public method that returns a byte array.
State
Returns the state of the hash computation. This property will contain a zero before the computations and a non-zero value after a successful hash computation. State is a protected field that returns an integer value representing the current state of the hash value computation.
Let's see a simple example to compute an MD5 hash. This method takes a byte array (clear text) and returns a byte array (hash value):
byte[] ComputeMD5(byte [] input)
{
MD5CryptoServiceProvider md5Provider = new MD5CryptoServiceProvider();
return md5Provider.ComputeHash(input);
}
As you can see, the MD5 provider is very easy to use. All we have to do is create a new object of the type MD5CryptoServiceProvider and pass the byte array to the ComputeHash() method, which returns a byte array. The same technique can be used for all the hash algorithms available in the .NET Framework. Let's write a simple Windows application to hash a string using multiple hash algorithms. Here is the application in action:
The implementation of the application is quite simple. We accept a string from the user and hash it using different hash algorithms. Here is the code for the Compute Hash button's Click event handler:
private void btnCompute_Click(object sender, System.EventArgs e)
{
if (txtHash.Text.Trim() != "")
{
// Generate bytes for the input string
byte[] inputData = ASCIIEncoding.ASCII.GetBytes(txtHash.Text);
// Display the hash value in textbox
txtMDS.Text = ASCIIEncoding.ASCII.GetString(new
MD5CryptoServiceProvider().ComputeHash(inputData));
txtSHAl.Text = ASCIIEncoding.ASCII.GetString(new
SHA1Managed().ComputeHash(inputData));
txtSHA256.Text = ASCIIEncoding.ASCII.GetString(new
SHA256Managed().ComputeHash(inputData));
txtSHA384.Text = ASCIIEncoding.ASCII.Getstring(new
SHA384Managed().ComputeHash(inputData));
txtSHA512.Text = ASCIIEncoding.ASCII.Getstring(new
SHA512Managed().ComputeHash(inputData));
}
}
First, we call the GetBytes() method of the ASCIIEncoding class to convert the string variable into a byte array. Then, we create a new object for each algorithm, and pass the input byte array to the ComputeHash() method. Then, we call the GetString() method of the ASCIIEncoding class again to convert the byte array into a string. Quite simple, isn't it-don't forget to add using directives for the System.Text namespace and System.Security.Cryptography namespaces to run this code.
Important If you're dealing with non-ASCII strings, you can use the UnicodeEncoding class in the System.Text namespace.
Using Hash Values for Authentication
Hashing techniques are very useful when it comes to authenticating users. We're going to see a simple authentication method for a Windows application using the MD5 algorithm. The username and the password supplied by the user will be authenticated against, for the purposes of illustration, an Access database.
Let's create an Access database called WroxDBAuth.mdb with a single table called Tbl_MA_Users. The table is going to store the user ID, e-mail address, password, first name, and last name, as shown below:
The e-mail address will be the login name for the users, and the password is stored in the database in the MD5 hash format.
This assures the users that their password can't be hacked. For example, the user's password "MyPass" is stored in the database as "{?*H!8*XJ~*N"H*S" in MD5 hash format. Let's build a simple authentication screen to authenticate users against the Access database:
Here is the code for the Login button:
private void btnLogin_Click(object sender, System.EventArgs e)
{
if (txtEmail.Text.Trim() != "" && txtPwd.Text.Trim() != "")
authenticateUser();
}
First, we check that something was entered in the username and the password text boxes. If something was entered, we call the authenticateUser() method. In the authenticateUser() method, we're connecting to the Access database and querying the table that matches the login name entered by the users.
eprivate bool authenticateUser()
{
bool bRtnValue = false;
string strConn = "PROVIDER=Microsoft.Jet.OLEDB.4.0;" +
"DATA SOURCE=c:\\DB\\WroxDBAuth.mdb;";
OleDbConnection Conn = new OleDbConnection(strConn) ;
Conn.Open();
String strSQL = "SELECT Pwd FROM Tbl_MA_Users WHERE Email = '" +
txtEmail.Text + "'";
OleDbCommand Cmd = new OleDbCommand(strSQL,Conn);
//Create a datareader, connection object
OleDbDataReader Dr = Cmd.ExecuteReader(
System.Data.CommandBehavior.CloseConnection);
//Get the first row and check the password.
if (Dr.Read())
(
Next, we pass the clear text password entered by the user into the GenerateMD5Hash() method, which returns the hashed string. If the current hashed password stored in the database and the hash generated by the GenerateMD5Hash() method are the same, we display the message Password was successful!; otherwise we displaying the message Invalid password:
if (Dr["Pwd"].ToString() == GenerateMD5Hash(txtPwd.Text))
{
MessageBox.Show(this,"Password was successful!");
bRtnValue = true;
}
else
{
MessageBox.Show(this,"Invalid password.");
}
}
else
{
MessageBox.Show(this,"Login name not found.");
}
Dr.Close();
return bRtnValue;
}
The GenerateMD5Hash() method is very simple. First, we convert the input string into a byte array using the ASCIIEncoding class. Next, we create a new object of type MD5CryptoServiceProvider and call its ComputeHash() method to generate the hash value. Then, we convert the hash value into a string and send it back to the caller:
string GenerateMD5Hash(string input)
{
// Generate bytes for the input string
byte[] inputData = ASCIIEncoding.ASCII.GetBytes(input);
// Compute the MD5 hash
MD5 md5Provider = new MD5CryptoServiceProvider();
byte[] hashResult = md5Provider.ComputeHash(inputData);
return ASCIIEncoding.ASCII.GetString(hashResult);
}
This is a simple procedure to implement, and it gives us an excellent security model for applications. In this example, we've used the MD5 algorithm. In the same way, we could use any of the other hash algorithms such as SHA1 to implement the application.
Important The only problem with this approach is that if the user wants his or her password e-mailed back to him or her, we won't be able to do it, since we can't convert the hash value back to clear text. However, we can always reset the password and send the new password back to the user.
Keyed Hash Values
The keyed hash or HMAC (Hash Message Authentication Code) algorithms are very similar to the hash algorithms, except that they generate the hash values based on a key. The HMAC algorithms are useful in the same way as the hash algorithms. For example, an HMAC value can be used to verify the integrity of a message transmitted between two parties that agree on a shared secret key. This is similar to the symmetric algorithm.
HMAC combines the original message with the key to compute a hash value. The sender computes the HMAC of the clear text and sends the HMAC with the clear text. The recipient recalculates the HMAC using the clear text and the sender's copy of the key. If the computed HMAC matches with the one sent from the other end, then the recipient knows that the original message has not been modified, since the message digest hasn't changed. In this way, the receiver can test the authenticity of the transmission. HMACs are commonly used as digital signatures.
The .NET Framework supports the HMACSHA1 and MACTripleDES algorithms. The HMACSHA1 algorithm computes keyed hash values using the SHA1 algorithm, and the MACTripleDES algorithm computes it based on the Triple-DES algorithm. We're going to see a simple example on how to use the HMAC classes. We'll build a Windows application that shows the HMAC value for the given clear text value and key.
First, we convert the input string and the key into byte arrays. Then we create an object of type HMACSHA1 and pass this object into a CryptoStream object. Then, we use the standard stream operations to read the input array and close the stream. The Hash property of the HMACSHA1 object returns the HMAC value. The same process is repeated for the MACTripleDES algorithm:
void ProcessKeyedHash(string input, string key)
{
try
{
// Generate bytes for the input string
byte[] inputData = ASCIIEncoding.ASCII.GetBytes(input);
byte[] keyBytes = new byte[16];
keyBytes = ASCIIEncoding.ASCII.GetBytes(key);
// Compute HMACSHA1
HMACSHA1 hmac = new HMACSHA1(keyBytes);
CryptoStream cs = new CryptoStream(Stream.Null, hmac,
CryptoStreamMode.Write);
cs.Write(inputData, 0, inputData.Length);
cs.Close();
txtHMACSHA1.Text = ASCII Encoding.ASCII.GetString(hmac.Hash);
// Compute the MACTripleDES
MACTripleDES macTripleDES = new MACTripleDES(keyBytes);
txtMACTripleDES.Text = ASCIIEncoding.ASCII.GetString(
macTripleDES.ComputeHash(inputData));
}
catch (Exception e)
{
MessageBox.Show(this, e.ToString());
}
}
Here is the application in action:
The only difference between these two algorithms is that the HMACSHA1 algorithm accepts keys of any size, and produces a hash value that is 20 bytes long. On the other hand, the MACTripleDES algorithm uses key lengths of 8, 16, or 24 bytes, and produces a hash value eight bytes long. If the key length is different from the requirement then an exception is thrown.
Symmetric Transformation with .NET
The System.Security.Cryptography namespace supports the DES, Triple-DES, RC2, and Rijndael symmetric algorithms. In this list, only the Rijndael algorithm is a managed implementation; the other algorithms use their counterparts in the Microsoft Crypto API.
The SymmetricAlgorithm Class
All the symmetric algorithm classes are inherited from the SymmetricAlgorithm class. The SymmetricAlgorithm class exposes some common methods and properties that can be used across all the hashing algorithms. The following table discusses a few of them:
Class Member
Description
Key
KeyValue
Specifies the secret key for the symmetric algorithm. Key is a public property and KeyValue is a protected field-both return a byte array.
KeySize
KeySizeValue
Specifies the size of the secret key in bits. KeySize is a public property and the KeySizeValue is a protected field-both return an integer value representing the length of the key in bits.
LegalKeySizes
LegalKeySizesValue
Specifies the valid key sizes in bytes for the current symmetric algorithm. LegalKeySizes is a public property and LegalKeySizesValue is a protected field-both return a KeySizes array.
IV
IVValue
Specifies the initialization vector for the symmetric algorithm. IV is a public property and IVValue is a protected field-both return a byte array.
BlockSize
BlocksizeValue
Specifies the block size in bits for the current symmetric algorithm. BlockSize is a public property and BlockSizeValue is a protected field-both return an integer.
LegalBlockSizes
LegalBlockSizesValue
Specifies the valid block size supported by the current symmetric algorithm. LegalBlockSizes is a public property and LegalBlockSizesValue is a protected field-both return a KeySizes array.
Mode
ModeValue
Specifies the mode of symmetric operation used by the current algorithm. Mode is a public property and ModeValue is a protected field-both return a CipherMode.
Padding
PaddingValue
Specifies the padding mode used by the current symmetric algorithm. Padding is a public property and PaddingValue is a protected field-both return a PaddingMode.
CreateEncryptor()
The CreateEncryptor() method creates a symmetric encryption object using the key and the initialization vector specified. CreateEncryptor() is a public method and returns an ICryptoTransform interface.
CreateDecryptor()
The CreateDecryptor() method creates a symmetric decryption object using the key and the initialization vector specified. CreateDecryptor() is a public method and returns an ICryptoTransform interface.
GenerateKey()
The GenerateKey() method generates a random key for the symmetric algorithm and overrides the value stored in the Key property. GenerateKey() is a public method and returns a random key in a byte array.
GenerateIV()
The GenerateIV() method generates a random initialization vector for the symmetric algorithm and overrides the value stored in the IV property. GenerateIV() is a public method and returns a random vector in a byte array
ValidKeySize()
Indicates whether the specified key size is valid for the current symmetric algorithm. ValidKeySizes() is a public method and returns an integer.
Let's start out by exploring the symmetric algorithms using the DES algorithm. Since symmetric algorithms tend to be faster than asymmetric ones, symmetric algorithms are good candidates for bulk encryption/decryption operations such as encrypting and decrypting entire files. We'll write a Windows application that will encrypt and decrypt files using the DES algorithm.
The user interface will provide options for locating a file using the Windows Common Dialog controls. There will be options provided to encrypt and decrypt a file with a secret key.
The Encrypt button will add a .enc extension to the source file when generating the encrypted destination file. Here is what the code looks like for the Encrypt button.
private void button1_Click(object sender, System.EventArgs e)
{
if (encryptData(encFile.Text, encFile.Text + ".enc" , txtKey.Text) == true)
MessageBox.Show(this, "Done!", "Encryption Status", MessageBoxButtons.OK,
MessageBoxIcon.Information);
else
MessageBox.Show(this, "The encryption process failed!", "Fatal Error",
MessageBoxButtons.OK, MessageBoxIcon.Stop);
}
The Encrypt button event calls the encryptData() method, passing in the source filename, destination filename, and the secret key for the encryption operation. Let's take look at the encryptData() method.
First, we create an object type of DESCryptoServiceProvider and assign the secret key supplied by the user to the Key property. Then we call the GenerateIV() method to generate an initialization vector for the encryption operation. Next, we create the DES encryption object by calling the CreateEncryptor() method of the DESCryptoServiceProvider class. After that, we instantiate two FileStream objects, one in read mode (the source file), and the other in write mode (the destination file) to encrypt the file:
// The encryptData method will encrypt the given file using the DES algorithm
public bool encryptData(string sourceFile, string destinationFile,
string cryptoKey)
{
try
{
// Create the DES Service Provider object and assign the key and vector
DESCryptoServiceProvider DESProvider = new DESCryptoServiceProvider();
DESProvider.Key = ASCIIEncoding.ASCII.GetBytes(cryptoKey);
DESProvider.GenerateIV();
ICryptoTransform DESEncrypt = DESProvider.CreateEncryptor();
// Open the source and destination file using the file stream object
FileStream inFileStream = new FileStream(sourceFile,
FileMode.Open, FileAccess.Read);
FileStream outFileStream = new FileStream(destinationFile, FileMode.Create,
FileAccess.Write);
The initialization vector (or IV) is always used to initialize the first block of plain text for encryption. We‘ve already talked about this in the Modes section.
Once we've created these objects, we need a CryptoStream object to which we write the encrypted file. We pass the DES encryption object and the output file stream into the CryptoStream's constructor. Then we read the content of the input file and write it back into the CryptoStream. Then, we close all the stream objects and return true:
// Create a CrytoStream class and write the encrypted out
CryptoStream CryptoStream = new CryptoStream(outFileStream, DESEncrypt,
CryptoStreamMode.Write);
// Declare the byte array of the length of the input file
byte[] bytearrayinput = new byte[inFileStream.Length - 1];
// Read the input file stream in to the byte array and write
//it back in the CryptoStream
inFileStream.Read(bytearrayinput, 0, bytearrayinput.Length);
CryptoStream.Write(bytearrayinput, 0, bytearrayinput.Length);
// Close the stream handlers
CryptoStream.Close();
inFileStream.Close();
outFileStream.Close();
return true;
}
catch (Exception e)
{
MessageBox.Show(this, e.ToString(), "Encryption Error",
MessageBoxButtons.OK, MessageBoxIcon.Stop);
return false;
}
}
The decryption process does the opposite of the encryption process. Let's look at the Decrypt button code. First, we retrieve the original filename from the selected one by removing the .enc extension from the filename. Then we call the decryptData() method with the source, destination, and the secret key.
private void button2_Click(object sender, System.EventArgs e)
{
string decFileHame = decFile.Text.Replace(".enc", "");
if (decryptData(decFile.Text, decFileName, txtKey.Text) == true)
MessageBox.Show(this, "Done!", "Decryption Status", MessageBoxButtons.OK,
MessageBoxIcon.Information);
else
MessageBox.Show(this, "The decryption process failed!", "Fatal Error",
MessageBoxButtons.OK, MessageBoxIcon.Stop);
}
The decryptData() method works in a very similar way to encryptData(). We create an object of type DESCryptoServiceProvider and assign the key to it. Then, we generate a new VI and create a new DES decryption object by calling the CreateDecryptor() method. Next, we read the source file into the CrytoStream and transform the content into a new file. Finally, we close the stream objects and return true:
// The decryptData method will decrypt the given file using the DES algorithm
public bool decryptData(string sourceFile, string destinationFile,
string cryptoKey)
{
try
{
// Create the DES Service Provider object and assign the key and vector
DESCryptoServiceProvider DESProvider = new DESCryptoServiceProvider();
DESProvider.Key = ASCIIEncoding.ASCII.GetBytes(cryptoKey);
DESProvider.GenerateIV();
FileStream DecryptedFile = new FileStream(sourceFile, FileMode.Open,
FileAccess.Read);
ICryptoTransform desDecrypt = DESProvider.CreateDecryptor();
CryptoStream cryptostreamDecr = new CryptoStream(DecryptedFile, desDecrypt,
CryptoStreamMode.Read);
StreamWriter DecryptedOutput = new StreamWriter(destinationFile);
DecryptedOutput.Write(new StreamReader(cryptostreamDecr).ReadToEnd());
DecryptedOutput.Flush();
DecryptedOutput.Close();
return true;
}
catch (Exception e)
{
MessageBox.Show(this, e.ToString(), "Decryption Error",
MessageBoxButtons.OK, MessageBoxIcon.Stop);
return false;
}
}
Using Other Symmetric Algorithms
Since all the symmetric algorithms are derived from the SymmetricAlgorithm class, it is very easy to implement the encryption/decryption process with the previous code base. For example, if you want to use the RC2, Triple-DES, or Rijndael algorithm, all you've to do is replace the following declaration in the encryptData() and decryptData() methods with the appropriate declarations shown below:
DES
DESCryptoServiceProvider DESProvider = new DESCryptoServiceProvider();
RC2
RC2CryptoServiceProvider RC2Provider = new RC2CryptoServiceProvider();
Triple-DES
TripleDESCryptoServiceProvider tDESProvider = new
TripleDESCryptoServiceProvider();
Rijndael
RijndaelManaged RijndaelProvider = new RijndaelManaged();
If you make this change, our file encrypter/decrypter application will work fine.
The success of the symmetric encryption and decryption process is based on the key value. If you don't supply a proper key length to the algorithm, then a CryptographicException will be raised. The key sizes supported by the algorithm can be fetched by accessing the LegalKeySizes property. This property returns an array of KeySizes objects. The KeySizes class has three integer public properties -MaxSize, MinSize, and SkipSize. The MaxSize and MinSize properties specify the maximum key size (in bits) and the minimum key size (in bits) respectively. The SkipSize returns the interval between the valid key sizes in bits.
The following table lists the key sizes supported by the major algorithms:
Algorithm
Key Size
DES
64 bits or 8 bytes
RC2
128 bits or 16 bytes
Triple-DES
192 bits or 24 bytes
Rijndael
256 bits or 32 bytes
The strength of the encryption is also based on the key. The larger the key, the better the encryption. Thus the likelihood of a hacker being able to decrypt the data with a brute-force attack is decreased. However, there is one more constraint that we should remember-the bigger the key, the more time required for the encryption and decryption process.
Asymmetric Transformation with .NET
As we discussed earlier in the chapter, asymmetric algorithms are based on the concept of public and private keys (PKI). The System.Security.Cryptography namespace supports two asymmetric algorithms: RSA and DSA.
The AsymmetricAlgorithm Class
Both the RSA and DSA algorithms inherit from the base class AsymmetricAlgorithm. The AsymmetricAlgorithm class exposes some common methods and properties that can be used across all the hashing algorithms:
Class Member
Description
KeySize
KeySizeValue
Specifies the size of the key modules in bits. KeySize is a public property and KeySizeValue is a protected field-both return an integer value representing the length of the key in bits.
LegalKeySizes
LegalKeySizesValue
Specifies a valid key size in bytes for the current asymmetric algorithm. LegalKeySizes is a public property and LegalKeySizesValue is a protected field-both return a KeySizes array.
KeyExchangeAlgorithm
Specifies the key exchange algorithm used when communicating between two ends and the way the public key and private key will be exchanged. This is a public property that returns a string representing the name of the key exchange algorithm used.
SignatureAlgorithm
Specifies the name of the algorithm used to sign the current object. This is a public property that returns a string representing the name of the signature algorithm used.
FromXmlString()
Reconstructs an Asymmetric object from an XML file. This is a public method that takes a string as input.
ToXmlString()
Returns an XML representation of the current algorithm object. This is a public method that returns a string.
As we've already discussed, many of the cryptography algorithms are implemented on top of the Crypto API library. The .NET Framework wraps the Crypto API library with sets of managed classes, called Cryptographic Service Providers (or CSPs). The CspParameters class is used to send values to and receive values from the unmanaged Crypto API.
The Cryptographic Service Providers are plug-ins for the Crypto API. These plug-ins are encryption engines that perform the encryption/decryption process.
The CSP operation is based on an enumeration value called CspProviderFlags. The CspProviderFlags enumeration supports two values: UseDefaultKeyContainer and UseMachineKeyStore. If the UseDefaultKeyContainer option is specified, the key information is read from the default key container. If the UseMachineKeyStore option is specified, the key information is read from the computer's key container.
CSPs maintain a database to store public/private key pairs. Some CSPs maintain their key container in the Registry, whereas others maintain it in other locations, such as in Smart Cards or encrypted, hidden files.
Using the RSA Algorithm
The RSA algorithm is implemented in the RSACryptoServiceProvider class, which inherits from the RSA class. The RSA class inherits from the AsymmetricAlgorithm class. The RSA algorithm allows us to encrypt, decrypt, sign data with a digital signature, and verify the signature. We'll look at these one by one in this section.
Let's start with a simple encryption/decryption approach. Since the RSA algorithm is an asymmetric algorithm, it is slower than its symmetric counterpart. Therefore, the RSA algorithm is best for small amounts of message encryption. Accordingly, we'll build a Windows application that will encrypt and decrypt the message entered by the user using PKI technology.
Whenever you create a new default constructor instance of the RSACryptoServiceProvider class, it automatically creates a new set of public/private key information, ready to use. We can also store the PKI values into XML files. This demonstration will be our first example. Let's build a UI as shown below.
This UI allows us to see the clear text as well as the encrypted cipher text. Our code will also show the public key and the private key used in this process and we'll have an option to store the public and private keys in different XML files.
Since we're going to use the RSA auto generated public/private keys, we declare a class level static object.
static RSACryptoServiceProvider objRSAProvider;
Here is the code for the Encrypt button:
if (txtClearText.Text.Trim() != "")
{
// Initialize the RSA Cryptography Service Provider (CSP)
rsaProvider = new RSACryptoServiceProvider();
UTFSEncoding utf8 = new UTFSEncoding();
byte[] clearText = utf8.GetBytes(txtClearText.Text.Trim());
// Encrypting the data received
txtCipherText.Text = Convert.ToBase64String(rsaProvider.Encrypt(clearText,
false));
// Show the private key
txtPrvKey.Text = rsaProvider.ToXmlString{true);
// Show the public Key
txtPubKey.Text = rsaProvider.ToXmlString(false);
}
We create a new object type of RSACryptoServiceProvider. Then we transform the user-entered message into a byte array using the UTF8Encoding class. Then we call the Encrypt() method of the RSACryptoServiceProvider class, pass the input byte array and a second parameter of false.
The second parameter in the Encrypt() method deals with the mode of operation. If you are running Windows 2000 OS with SP2 or higher then you can set this parameter to true, which will use the OAEPpadding method. When set to false it'll use PKCS version 1.5.
The Encrypt() method returns an encrypted byte array. We're using the ToBase64String() method of the Convert class to convert the byte array into a string for display in the textbox. We also display the private/public key generated by the RSA algorithm in two text boxes using the ToXmlString() method of the RSACryptoServiceProvider class. ToXmlString() takes a Boolean as input and if the value is false, it generates the public key value as an XML string; if the value is true, it generates the private key.
Here is the code for our Decrypt button. This method just calls the Decrypt() method of the RSACryptoServiceProvider object:
private void btnDecrypt_Click(object sender, Systern.EventArgs e)
{
if (txtCipherText.Text.Trim() ! = "")
{
//Convert the input string into a byte array
byte[] bCipherText =
Convert.FromBase64String(txtCipherText.Text.Trim());
//Decrypt the data and convert it back to a string
string strValue =
ASCIIEncoding.ASCII.GetString(objRSAProvider.Decrypt(
bCipherText, false));
//Display the decrypted string in a MessageBox
MessageBox.Show(this, strValue, "Decrypted value",
MessageBoxButtons.OK,MessageBoxIcon.Information);
}
}
Now let's look at saving the public/private key into an XML file. We display the SaveFile common dialog box to get the desired filename from the user. Then we load the XML data into an XmlDocument object and call its Save() method to save the data to disk:
private void btnSave_Click(object sender, System.EventArgs e)
{
SaveFileDialog saveFileDialog1 = new SaveFileDialog();
saveFileDialogl.Filter = "XML files (*.xml)|*.xml|All files(*.*)|*.*" ;
saveFileDialog1.FilterIndex = 2 ;
saveFileDialog1.RestoreDirectory = true ;
if(saveFileDialog1.ShowDialog() == DialogResult.OK)
{
// Write the content to an XML file
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.LoadXml(this.txtPubKey.Text);
// Save the document to a file.
xmlDoc.Save(saveFileDialog1.OpenFile());
}
}
Here is the application in action:
As you can see, the clear text, encrypted cipher text, and the decrypted clear text is shown on the screen with the public and private key information.
Loading the Public and Private Keys
In the previous example, we saw how to encrypt and decrypt data using the auto-generated public and private keys. We also saw how to save the keys into an XML file. However, if you want to reuse previously created or saved keys, you can do this by initializing the class with a populated CspParameters object. Let's see an example of this. Create a UI like the previous one, but this time we'll load the public and private keys from the XML files:
Here is the code to load a key from XML. First, we display an OpenFile dialog to get the filename from the user. Then, we load the file into an XmlTextReader object and display the key information in the appropriate text box:
private void btnLoadPub_Click(object sender, Systern.EventArgs e)
{
// Show the open file dialog
openFileDialog1.Title = "Select the Public Key file";
openFileDialog1.Filter = "XML Files (*.xml)|*.xml";
if(openFileDialog1.ShowDialog() == DialogResult.OK)
{
string fileName = openFileDialog1.FileName;
btnEncryptl.Enabled = true;
// Load the document
Xml Text Reader xmlReader = new XmlTextReader(fileName);
xmlReader.WhitespaceHandling = WhitespaceHandling.None;
xmlReader.Read();
// Assign the public key to the textbox
txtPubKeyl.Text = xmlReader.ReadOuterXml();
}
}
Now let's look at the code behind the Encrypt button. First, we create an object of the CspParameters class and set the flag to use the machine store to look for the PKI keys. Then we give the key container the name WroxRSAStore.
private void btnEncryptl_Click(object sender, System.EventArgs e)
{
if (txtClearText1.Text.Trim() != "")
{
try
{
CspParameters cspParam = new CspParameters();
csp Param.Flags = CspProviderFlags.UseMachineKeyStore;
cspParam.KeyContainerName = "WroxRSAStore";
cspParam.ProviderName = "MS Strong Cryptographic Provider";
// CryptoAPI constant -> PROV_RSA_FULL = 1
// This provider type supports both digital
// signatures and data encryption, and is considered
// general purpose. The RSA public-key algorithm
// is used for all public-key operations.
cspParam.ProviderType = 1;
Once we've initialized the parameters for the CSP, we create a new object of type RSACryptoServiceProvider using our CspParameters object. Then, we assign the public key by calling the FromXml String() method. After that, we perform our usual process of converting the string into a byte array and passing the byte array into the Encrypt() method and converting the byte array back to a string:
// Initializing the RSA Cryptography Service Provider (CSP)
RSACryptoServiceProvider rsaProvider1 = new
RSACryptoServiceProvider(cspParam);
// Load the public key
rsaProvider1.FromXmlString(txtPubKey1.Text);
UTF8Encoding utf8 = new UTF8Encoding();
byte[] clearText = utf8.GetBytes(txtClearText1.Text);
// Convert encrypted text to base64
txtCipherText1.Text = Convert.ToBase64String(
rsaProvider1.Encrypt(clearText, false));
}
catch (Exception e)
{
MessageBox.Show(this, e.ToString());
}
}
}
The decryption method is again very simple-we just create an RSA CSP object and assign the private key to it. We then decrypt the message using the Decrypt() method:
private void btnDecrypt1_Click(object sender, System.EventArgs e)
{
if (txtClearText1.Text.Trim() != "")
{
// Initialize the RSA Cryptography Service Provider (CSP)
RSACryptoServiceProvider rsaProvider1 = new RSACryptoServiceProvider();
// Load the private key
rsaProvider1.FromXmlString(this.txtPriKey1.Text);
// Encrypt the data received
MessageBox.Show(this, ASCIIEncoding.ASCII.GetString(
rsaProvider1.Decrypt(Convert.FromBase64String(
txtCipherText1.Text.Trim()), false)), "Decrypted value",
MessageBoxButtons.OK, MessageBoxIcon.Information);
}
}
Here is the application in action:
There is one limitation of the RSA algorithm that you should know-the Encrypt() method can only encrypt up to 16 bytes if the High Encryption pack is installed. Otherwise, it can only encrypt 5 bytes.
Reading an X509 Certificate
A certificate is like a 'voucher' that contains information about the person holding the voucher, such as who authorized the certificate, its public keys, and its expiration information. Certificates are signed by a certifying authority (or CA), such as VeriSign or Thawte. The Microsoft Certificate Server also allows us to create 'self-signed' certificates. However, they may not be trusted by the rest of the world, since this is not issued by a well-known CA, but they are very useful in an intranet scenario.
Server certificates are used to identify the trustworthiness of the server, and client certificates to identify a client to the server. A CA issues both client and server certificates after verifying the identity. For example, when the client is requesting a web resource, it can also send a client certificate along with the request. The server can then determine who the client is and authorize or deny them.
Thawte provides the client certificate free of charge but VeriSign charges a fee for it.
Client certificates are usually installed on web clients such as browsers and e-mail clients. You can view all the client certificates installed in IE by clicking on Tools | Internet Options… and selecting the Content tab in the dialog box. Click on the Certificates… button here. As you can see, | have two client certificates installed in IE 6:
The Cryptography namespace also contains the child namespace X509Certificates. This contains just three classes used to represent and manage Authenticode X509 v.3 certificates. The X509Certificate class exposes the static methods CreateFromCertFile() and CreateFromSignedFile() to create an instance of the certificate.
The CreateFromCertFile() method reads the content of the X509 certificate from a certificate file and the CreateFromSignedFile() method reads the content of the X509 certificate from a Digitally Signed file. We will use the CreateFromCertFile() method to read the contents of the X509 client certificate:
private void btnView_Click(object sender, System.EventArgs e)
{
// Read the client certificate from the file
// into the object variable of the type X509Certificate
X509Certificate clientCert =
X509Certificate.CreateFromCertFile("C:\\test.cer");
StringBuilder sb = new StringBuilder();
sb.Append("Issuer Name: " + clientCert.GetIssuerName() + "\n");
sb.Append("Public Key String: " + clientCert.GetPublicKeyString() + "\n");
sb.Append{"Key Algorithm: " + clientCert.GetKeyAlgorithm().ToString() + "\n");
sb.Append("Serial Number: " + clientCert.GetSerialNumberString() + "\n");
sb.Append("Effective Date: " +
clientCert.GetEffectiveDateString().ToString() + "\n");
sb.Append("Expiration Date: " +
clientCert.GetExpirationDateString().ToString() + "\n");
MessageBox.Show(this, sb.ToString());
}
We've declared an object of type X509Certificate, and we've used the CreateFromCertFile() method to read the certificate content into the object. Then we've read the main properties of the certificate and disolaved it to a message box:
Cryptography and Network Programming
So far in this chapter we've learnt all about cryptography-now it's time to use some of the techniques in network programming. Remember the simple UDP chat utility that we wrote in Chapter 6? The UDP chat application is very simple. It takes the local and remote port numbers and the IP address and passes the information back and forth.
Here is the code before adding any cryptography algorithms. The Send() method here gets the datagram, converts it into a byte array, and sends the data using the Send() method of the UdpClient class. The Receiver() method here also works in the same way.
private static void Send(string datagram)
{
// Create UdpClient
UdpClient sender = new UdpClient();
// Create IPEndPoint with details of remote host
IPEndPoint endPoint = new IPEndPoint(remoteIPAddress, remotePort);
try
{
// Convert data to byte array
byte[] bytes = Encoding.ASCII.GetBytes(datagram);
// Send data
sender.Send(bytes, bytes.Length, endPoint);
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
finally
{
// Close connection
sender.Close();
}
}
public static void Receiver()
{
// Create a UdpClient for reading incoming data.
UdpClient receivingUdpClient = new UdpClient(localPort);
// IPEndPoint with remote host information
IPEndPoint RemoteIpEndPoint = null;
try
{
Console.WriteLine(
"-----------*******Ready for chat!!!*******-----------");
while(true)
{
// Wait for datagram
byte[] receiveBytes = receivingUdpClient.Receive(
ref RemoteIpEndPoint);
// Convert and display data
string returnData = Encoding.ASCII.GetString(receiveBytes);
Console.WriteLine("−" + returnData.ToString());
}
}
catch (Exception e)
{
Console.WriteLine(e.ToString ());
}
}
Let's use the Rijndael symmetric algorithm to send secure information between two chat clients. The Rijndael algorithm is a symmetric algorithm and it is best for bulk data transfer. This is the main reason that I've chosen this algorithm. We're going to add encryptData() and decryptData() methods to this class to take care of the secure communication process. When we're sending information to the remote socket we'll call the encryptData() method to encrypt the data, and then we'll pass the encrypted byte array to the other end. When we receive the encrypted byte array on the other end, we'll call the decryptData() method to decrypt the byte array.
As you can see, our design is very simple. To support the symmetric algorithms, we've added two private class level members that store the shared key and vector.
private static IPAddress remoteIPAddress;
private static int remotePort;
private static int localPort;
private static UTF8Encoding Utf8Encod;
private static string CryptoKey = "!i~6ox1i@]t2K'y$";
private static string CryptoVT = "!~x7Oq{6+q1@#VI$";
The encryptData() method takes a string as input and returns a byte array.
static byte[] encryptData(string theDataGram)
{
byte[] bCipherText = null;
try
{
We create a new object type of RijndaelManaged, and assign the shared key and vector. Then we create an ICryptoTransform object using the CreateEncryptor() method.
// Create the Rijndael Service Provider object and assign the
// key and vector to it
RijndaelManaged Rijndael Provider = new RijndaelManaged();
Rijndael Provider.Key = Utf8Encod.GetBytes(CryptoKey);
RijndaelProvider.IV = Utf8Encod.GetBytes(CryptoVI);
ICryptoTransform RijndaelEncrypt = Rijndael Provider.CreateEncryptor();
Now, we convert the datagram into a byte array using the UTF8Encoding class. Then we declare a MemoryStream object and use a CryptoStream object to perform the cryptographic transformation- you may recall that we had a fleeting look at the CryptoStream class in Chapter 2.
// Convert string to byte array
byte[] bClearText = Utf 8Encod.GetBytes{theDataGram);
MemoryStream Mstm = new MemoryStream();
//Create Crypto Stream that transforms a stream using the encryption
CryptoStream Cstm = new CryptoStream(Mstm, RijndaelEncrypt,
CryptoStreamMode.Write);
//Write out encrypted content into MemoryStream
Cstm.Write (bClearText, 0, bClearText.Length);
Ctms.FlushFinalBlock();
We create the byte array back from the MemoryStream and return the byte array to the caller.
//Get the output
bCipherText = Mstm.ToArray();
//Close the stream handlers
Cstm.Close();
Mstm.Close();
}
catch (Exception e)
{
Console.WriteLine(e.ToString ());
}
return bCipherText;
}
The decryptData() method, which does the decryption process, is similar to the encryptData() method. The decryptData() method takes a byte array as input and returns a string as output.
static string decryptData(byte[] bCipherText)
{
string sEncoded ="";
try
{
// Create the RijndaelManaged Service Provider object and assign
// the key and vector to it
RijndaelManaged RijndaelProvider = new RijndaelManaged();
RijndaelProvider.Key = Utf8Encod.GetBytes(strCryptoRey);
RijndaelProvider.IV = Utf8Encod.GetBytes(strCryptoVI);
ICryptoTransform RijndaelDecrypt= RijndaelProvider.CreateDecryptor();
//Create a MemoryStream with the input
MemoryStream Mstm = new MemoryStream(bCipherText, 0, bCipherText.Length);
//Create Crypto Stream that transforms a stream using the decryption
CryptoStream Cstm = new CryptoStream(Mstm, RijndaelDecrypt,
CryptoStreamMode.Read);
// read out the result from the Crypto stream
StreamReader Sr = new StreamReader(Cstm);
sEncoded = Sr.ReadToEnd();
Sr.Close();
Cstm.Close();
Mstm.Close();
}
catch (Exception e)
{
Console.WriteLine(e.ToString ());
}
return sEncoded;
}
Let's call the encryptData() and decryptData() methods in the proper places in the chat application. The Send() method calls the encryptData method, with the string entered by the user, and sends the result via the Send() method of the UdpClient. The Receiver() method passes the received byte array into the decryptData() method and displays the decrypted message.
private static void Send(string datagram)
{
...
try
{
// Convert string to byte array
//byte[] bClearText = Utf8Encod.GetBytes(datagram);
// Encrypting the data recived
byte[] bytes = encryptData(datagram);
// Send data
sender.Send(bytes, bytes.Length, endPoint);
}
...
}
public static void Receiver()
{
...
while(true)
{
// Wait for datagram
byte[] receiveBytes = receivingUdpClient.Receive(ref RemoteIpEndPoint);
//Decrypt the incoming byte array
string returnData = decryptData(receiveBytes);
Console.WriteLine("−" + returnData.ToString());
}
...
}
We can also implement this with an asymmetric algorithm such as RSA-let's see an example of this. We'll just rewrite the encryptData() and decryptData() methods using the RSA algorithm. The rest of the implementation will remain the same.
Before writing the encryptData() and decryptData() methods we've to store the public key and the private key in private class level variables.
private static IPAddress remoteIPAddress;
private static int remotePort;
private static int localPort;
private static UTF8Encoding Utf8Encod;
private static string PubKey =
"sttDL3xug/BqMk13d6G5vWekmyul/d3pz/Lpvk2Q1GNBSriatLxCRJSuOAi
e8g/yby624K85qJLwMMzwCru7b+kNTA2dYaK4Nk+FkZMLCVmomiW1zns2KsT1aF9hwr32Nyje3OuJDlHqB
tcOpCGbo+kJ+JC88BM1J9AkdoAa+SE= AQAB
private static string PriKey =
"sttDL3xug/BqMk13d6G5vWekmyul/d3pz/Lpvk2Q1GNBSriatLxCRJSuOAi
e8g/yby624K85qJLwMMzwCru7b+kNTA2dYaK4Nk+FkZMLCVmomiW1zns2KsT1aF9hwr32Nyje3OuJ01HqB
tcOpCGbo+kJ+JC88BM1J9AkdoAa+SE=AQAB bikCjwD+gPRs6KmJ0gCp6FOY4V0WYFWthNcLkQ1Y5zfsWsyrpP649tC/dGkwZpggY6CJ
GwcmBIAHa1hez2yJTQ== Uzva1Xkzpvuf+89xrcq9YQArwYDqmKGPLDy0cC2cxq6czarI+XRAy
guEeFYjp2RIatLMrcA4QV4KV3+DzQWaeQ== eriVG9Kp3CQ/J9PpfMlemC7tPIs6m//L
yhKD7J5zLGIzz+71C5QjVi2dRwtvjGJaexOTi+TRIv2fT/LhWmsCDQ== sjfHZ470tIuf
1gXY8AznfnLC05eXrDIuo/YBsY2qredFDQaLqWIZiiq4ur7kWoFHakAbHCGeC3p2+bmLyrYr2nm8OgjOc1
NUneE8ASoKWfnbcWxW377Oeogj16frPUoAgwU1gFURdTxozgNLThVtNItrc3Doa5eJ+U7pRSz2edE=
Here is how the encryptData() and decryptData() methods look. In the encryptData() method we create a new CspParameters object, and use the object to create a new RSA algorithm object. Next, we assign the public key from the private class member. Then, we call the Encrypt() method of the RSA object.
static byte[] encryptData(string strDataGram)
{
CspParameters theCspParam = new CspParameters();
theCspParam.Flags = CspProviderFlags.UseMachineKeyStore;
theCspParam.KeyContainerName = "WroxRSAStore";
// Initializing the RSA Cryptography Service Provider (CSP)
RSACryptoServiceProvider theRSAProvider = new
RSACryptoServiceProvider(objCspParam);
//Set the Load the public key
theRSAProvider.FromXmlString(PubKey);
// Convert string to byte array
byte[] bClearText = Utf8Encod.GetBytes(strDataGram);
// Encrypting the data received
byte[] bytes = theRSAProvider.Encrypt(bClearText, false);
theRSAProvider.Clear();
return bytes;
}
The decryptData() method does pretty much the same thing. It creates a new RSA object and assigns the private key from the class member. Then it calls the Decrypt() method and sends the string back to the caller.
static string decryptData(byte[] bCipherText)
{
// Initializing the RSA Cryptography Service Provider (CSP)
RSACryptoServiceProvider theRSAProvider = new RSACryptoServiceProvider();
//Load the private key
theRSAProvider.FromXmlString(PriKey);
// Encrypting the data received
string strRtnData = Utf8Encod.GetString(theRSAProvider.Decrypt(bCipherText,
false));
theRSAProvider.Clear();
return strRtnData;
}
This implementation of the encryptData() and decryptData() methods can be replaced with the previous example and the code will work fine. The only thing that we've to be careful about in this example is the size limitations of the Encrypt() method-we talked about these limitations earlier.
Cryptography is all about converting plain text or clear text into cipher text through a process known as encryption. The cipher text is converted back to plain text or clear text by the opposite process called decryption:
The mathematical cryptography algorithm that performs the encryption and decryption transformations is also called a cipher and the encrypted text is called the cipher text.
Let's look at a simple example. Let's build a simple ASP.NET page.
All this ASPX page does is to display 'Some Information!' in a label server control in the Page_Load() event. Pretty simple, isn't it? Let's see what the page looks like in IE 6:
Let's use a tool called TCPTrace.exe, which is a network-sniffing tool. Microsoft also ships a similar tool called Network Monitor with the Windows Server products.
Important The TCPTrace.EXE tool can be downloaded from www.PocketSOAP.com
My web server (IIS 5.0) is running on port 80 and in order to sniff the packets sent to my web server, we've to change the port on the TCPTrace utility to 81 and the TCPTrace utility will forward all the requests received at port 81 to port 80 (to the web server). When accessing the site from IE use port 81. For example, if you're accessing your localhost then you've to access it like this: http://localhost:81/
As you can see, anyone who has access to a simple network-sniffing tool can read the information that we've transmitted over the public network. So how do you prevent this information leak? Cryptography plays a major
Why Use Cryptography?
If your computer is connected to or transmits information over an electronic network your data is visible to everyone and is available for hackers. Today, more and more companies are doing business online and this increases the security risk for the companies online business as well as the customers and partners that interact with the company. To address these problems, each and every company must take strong steps to protect their online business as well as their customers and partners.
When used properly, cryptography addresses the following problems:
Confidentiality- confidentiality assures that your information is protected.
Authentication- authentication assures that you know who is accessing your private network.
Integrity- integrity assures that information is not being tampered with during transit.
Non-repudiation- non-repudiation assures that the sender can't deny sending the message.
Cryptography provides all the services to address the security and privacy concerns of transmitting sensitive data over a public network.
Concepts of Cryptography
A cryptography algorithm is a mathematical function that transforms the readable plain text message into unreadable text garbage, and reverses the process to produce readable text from an encrypted message. All cryptographic algorithms are based on two simple principles:
Substitution- the concept of substitution is based on simply replacing every character in the message with another one. For example, when using the Caesar cipher, for a given letter in the message, shift to the right (in the alphabet) by three. That is, an 'a' becomes 'd', 'b' becomes 'e', and so on. This can be generalized to work for any number n not greater than 25 (assuming a 26 letter alphabet). In this cipher, the number n is the 'key'.
There are many varieties of the substitution ciphers available, including mono-alphabetic substitution ciphers, poly-alphabetic substitution ciphers, and perfect substitution ciphers.
Transposition- the concept of transposition is based on scrambling the characters that are in the message. Some common forms of transposition algorithm involve writing the message into a table row-by-row and reading them column-by-column. Some of the transposition algorithms such as Triple-DES perform this process three times to create the cipher text.
The math formula of cryptography is simple. When you pass the plain text message into the encryption function, the function should provide the CipherMessage, and when you pass the CipherMessage into the decryption function, it should return the original message:
Encryption(Message) = CipherMessage
And:
Decryption(CipherMessage) = Message
In the same way, the following formula should also be true:
Decryption(Encryption(Message)) = Message
In the simple encryption/decryption algorithms, the algorithm is well known, and anyone who knows the algorithm should be able to decrypt the CipherMessage. Therefore, to provide more security for the algorithms, a key is added.
We need a key to lock and unlock a lock. In the same way, a cipher is a math algorithm and a key is a sequence of bytes that is used to encrypt and decrypt the information. If we don't have the key, we can't unlock the lock. In the same way, if we lose the key we can't decrypt the encrypted data. The keys come in different sizes based on the cryptography algorithms. For example, the DES algorithm is based on 56 bits and the RC2 algorithm is based on 128 bits.
In order to encrypt or decrypt the message we have to pass the appropriate key into the function:
Encryption(Message, Key) = CipherMessage
And:
Decryption(CipherMessage, Key) = Message
In the same way, the following formula should also be true:
Decryption(Encryption(Message, Key), Key) = Message
Cryptographic Algorithms
Cryptographic algorithms can be divided into three types:
Symmetric algorithms- in symmetric cryptographic algorithms, the same key is used for encrypting and decrypting the message.
Asymmetric algorithms- in asymmetric cryptography algorithms, different keys are used for encrypting and decrypting the messages. The asymmetric cryptography algorithms are also known as public key infrastructure (or PKI).
Hashing or Message Digest algorithms- in hash cryptographic algorithms, the original text is transformed into a fixed-length cipher text. The hash algorithms also perform one-way encryption meaning the hashed cipher text can't be decrypted to its original clear text version. The fixed length of the cipher text changes based on the algorithm from 128 to 256 bits.
Symmetric Algorithms
When encrypting and decrypting the information, the same key is used for the encrypting and decrypting process in symmetric algorithms, as shown in the following figure:
With this method, the data can be transmitted in an insecure network and the receiver can decrypt the information using the same cryptography algorithm used by the sender. Of course, the key that is used to encrypt and decrypt should be kept secret if you want this method to work for you. Another problem with this approach is distributing the key to the other end where it needs to be encrypted or decrypted.
For more information about the key sizes, read the paper 'Selecting Cryptographic Key Sizes' at www.Cryptosavvy.com/Cryptosizes.pdf.
Some of the common symmetric encryption algorithms are:
DES- the Data Encryption Standard was adopted by the US Government in 1977, and by ANSI in 1981. DES follows 56-bit key for encryption and decryption. DES is a very famous algorithm but due to its small key length support, its use is very limited in today's world.
TripIe-DES- Triple-DES (or 3DES) is a very secure algorithm when compared with DES, since Triple-DES encrypts the message three times using the DES algorithm with different keys. The total key length of Triple-DES is 168 bits.
Blowfish- Blowfish is a fast, compact, and simple encryption algorithm invented by the famous author Bruce Schneier, who is the author of the celebrated book 'Applied Cryptography' (John Wiley & Sons, ISBN 0471128457). This algorithm allows a variable key length up to 448 bits.
IDEA- the International Data Encryption Algorithm (IDEA) was developed by James L. Massey and Xuejia Lai in Switzerland. Widespread use of this algorithm was hindered by several patent problems.
RC2, RC4, RC5- the RC2 and RC4 algorithms were originally developed by Ronald Rivest for RSA Security. Both RC2 and RC4 allow key lengths between 1 and 2,048 bits. On the other hand, RC5 allows a user-defined key length, data block size, and number of encryption rounds.
Rijndael (AES)- this algorithm was originally developed by Joan Daemen and Vincent Rijmen. Rijndael is a fast and compact algorithm that supports keys of 128, 192, and 256 bits in length.
Important The .NET Framework supports DES, Triple-DES, RC2 and Rijndael symmetric encryption algorithms.
Symmetric key algorithms are much faster than PKI algorithms, so they are the preferred choice for encrypting and decrypting large blocks of data. Also, they're also very easy to implement. The main disadvantage of symmetric encryption is that we need to protect the keys, and it is a challenge to exchange the keys between the encryption source and the decryption destination. The security of the algorithms is also related to the length of the key: the longer the key, the slimmer the probability of the information being decrypted. The possibility of information being decrypted is also based on the complexity of the key that you've chosen. The more complex the key, the slimmer the probability of the information being decrypted.
Asymmetric Algorithms
With asymmetric algorithms, one key encrypts the message and the other key decrypts the information as shown in the following figure:
Both the keys are different but they're related to each other. Therefore, we can publish the public key without worrying about the possibility of our encryption being compromised. This encryption is also called public-key encryption, or Public Key Infrastructure (PKI), since the public key is available publicly without compromising the integrity and security of the key or message. The decryption key is normally called the private key or secret key.
Public-key cryptography and related standards and techniques underlie security features of many products, including signed and encrypted e-mail, form signing, object signing, single sign-on, and the most popular Secure Sockets Layer (SSL)/TLS protocol.
As shown in the figure above, the public key can be freely distributed, but only you will be able to read a message encrypted using this key. When someone sends an encrypted message to you, they encrypt the message with your public key, and upon receiving the encrypted data you can decrypt it with the corresponding private key. The message encrypted with the public key can be decrypted only with the corresponding private key.
Important The PKI systems also use key exchange methods such as 'Diffie-Hellman key exchange'. The Diffie-Hellman key exchange is not an algorithm-it is a method that allows us to develop secure key exchange between two parties. The other methods are Digital Signature Standard (or DSS) and Elliptic Curve Cryptosystems.
The other way is to download the public key from an online repository. For example, if you get a client certificate from Thawte, you've an option to add your public key to the online repository. When someone wants to send something very secure all they've to do is to get the public key from you or from the online repository and sign the information with your public key and send it to you. When you receive the message, you decrypt it with your private key. This is one of the most common ways of exchanging the key.
Compared with symmetric-key encryption, public-key encryption requires more computation and is therefore not always appropriate for large amounts of data.
The .NET Framework supports two asymmetric algorithms:
DSA/DSS- Digital Signature Standard (DSS) was developed by the National Security Agency. DSS is based on the Digital Signature Algorithm (DSA), and it supports any key length.
RSA- RSA is a well-known public key algorithm developed by Ronald Rivest, Adi Shamir, and Leonard Adleman, and supports variable key length based on the implementation.
Message Digest Algorithms
Message Digest Algorithms (also known as MAC or hash algorithms) transform a variable-size input and return a fixed-size string as shown in the following figure:
The hash algorithms are also called one-way hash, since the hashed string can't be converted back to the original state-once the original clear text value has been hashed, it is not possible to get the original hashed value from the MAC.
When we hash a clear text message with a hash algorithm, it uses a key to produce the hash value. Once the hash value is generated, we can send the clear text and the hash to the other end, where the clear text value can be hashed with the same key and algorithm-the generated hash value can then be compared with the one supplied by the other end. If the values match, we can be sure that the message has not been altered in transit.
Important ASP.NET's Forms authentication module supports the one-way hash algorithms MD5 or SHA3 to authenticate the usernames/passwords stored in the config.web file. The same functionality can also be extended by storing the hashed passwords in the database using the HashPasswordForStoringInConfigFile method of the FormsAuthentication class available in the System.Web.Security namespace. For more information, have a look at 'Professional ASP.NET Security' (Wrox Press, ISBN: 1-86100-620-9).
Some of the common MAC functions are MD2, MD4, MD5, SHA, SHA-1, SHA-256, SHA-384, and SHA-512. The .NET Framework supports the following hash functions:
HMACSHA-1
MACTripleDES
MD-5
SHA-1
SHA-256
SHA-384
SHA-512
A Hash Message Authentication Code (or HMAC) function is a technique for verifying the integrity of a message transmitted between two parties that agree on a shared key.
Digital Signatures
Although encryption and decryption address a few problems, there are two important problems they don't address:
Tampering
Impersonation
Digital signatures use the one-way hashing functions for tamper detection and related authentication problems. Since the value of the hash is unique for the hashed data, any change in the data, even deleting or altering a single character, results in a different value. Moreover, the content of the hashed data cannot, for all practical purposes, be deduced from the hash. This therefore becomes the best way to detect tampering.
In public key encryption, it's possible to use the private key for encryption and the public key for decryption. Since this could create problems when encrypting sensitive information, we can digitally sign any data, instead of encrypting the data itself. Signing the data creates a one-way hash of the data, which can then be encrypted using the private key. The encrypted hash, along with other information, such as the hashing algorithm, is known as a digital signature. The following figure shows a simplified view of the way a digital signature can be used to validate the integrity of signed data:
If you look at the above figure, the sender sends the clear text (or encrypted) message with the Digital Signature. The Digital Signature is computed based on the clear text message and the clear text message is hashed using a hashing algorithm such as MD5 and the hashed value will be signed by the private key. At the other end, we'll receive the clear text message (or encrypted) with the digital signature. Then we'll compute a hash value for the clear text and compare it with the Digital Signature. If the Digital Signature verification process was successful then, we're assured that the data has not been tampered with in transit.
Cryptography Terminology
Before diving into the world of crypto coding, we need to understand some cryptographic terminology.
Block Ciphers and Stream Ciphers
Cryptographic ciphers handle data in two formats:
Block ciphers
Stream ciphers
Block ciphers are traditionally the most popular ones. A block cipher transforms a fixed-length block of plain text data into a block of cipher text data of the same length and then repeats the process until the entire message has been processed. This transformation takes place under the action of a user-provided secret key. Decryption is performed by applying the reverse transformation to the cipher text block using the same secret key. The fixed length is called the block size, and for many block ciphers, the block size is 64 bits. Typically, symmetric algorithms are based on the block cipher format. For example, the DES and RC2 algorithms use 8 bytes, 3DES uses 16 bytes, and Rijndael uses 32 bytes as input. Using this scale each algorithm splits the input into the blocks and performs the transformation.
More recent symmetric encryption algorithms are based on stream ciphers. Every stream cipher generates a keystream and encryption is provided by combining the keystream with the plain text (usually with the bitwise XOR operator). Stream ciphers can be designed to be exceptionally fast, much faster in fact than any block cipher. While block ciphers operate on large blocks of data, stream ciphers typically operate on smaller units of plain text, usually bits. The encryption of any particular plain text with a block cipher will result in the same cipher text when the same key is used. With a stream cipher, the transformation of these smaller plain text units will vary, depending on when they are encountered during the encryption process.
Padding
Block ciphers deal with blocks of bits (usually 64 bits), and the last remaining bits may not fit in a block. For example, suppose we have 136 bits of information that we are trying to encrypt using a block cipher that takes 64 bits (or 8 bytes) at a time to encrypt. The following figure shows how the 136-bits input is split into 64-bit blocks for the padding process.
In the process of encryption, the first two blocks will contain 128 bits, and the remaining eight bits will not fit in the block cipher's buffer. To address the incomplete block, padding is needed. A padding scheme will define how the last incomplete block of data will be handled in the process of encryption. The padding will be addressed in the process of decryption by removing all the padded characters and restoring the original text.
PKCS#5 (or Public Key Cryptography Standard) is one of the most famous padding schemas and it was published by RSA Security, Inc. For more information, please visit the RSA web site at http://www.rsa.com/rsalabs/pubs/PKCS/.
The .NET Framework handles padding using the PaddingMode enumeration. The PaddingMode enumeration supports three values:
None
PKCS7
Zeros
As the name suggests, when we use None, no padding is done. When PKCS7 is used, the remaining number of blocks will be filled with the remaining number of bits. For example, if six bits are free in a given byte, the last six bits will be padded with the value six. If four bits are free, the last four bits will be padded with the value four:
When PaddingMode.Zeros is used, the remaining bytes will be filled with the value zero:
Modes
The mode of a cipher determines how blocks of plain text will be encrypted into blocks of cipher text, and decrypted back. The CipherMode enumeration defines the block cipher mode to be used when performing the encryption or decryption process. You can specify the mode using the Mode property of many of the cipher algorithms. The .NET Framework supports CBC, CFB, CTS, ECB, and OFB modes.
When using ECB (or Electronic Code Book) mode, each block of plain text is encrypted to a block of cipher text. The main drawback to ECB mode is that the same plain text will always encrypt to the same cipher text when the same key is used.
The CBC (or Cipher Block Chaining) mode overcomes the drawbacks of ECB mode. When using CBC mode, each block of plain text is combined with the previous block's cipher text (using an XOR operation), which produces encrypted blocks of cipher text:
Since there is no previous cipher text block when starting the process, an initialization vector (or IV) is used for the first block of plain text.
The CFB (or Cipher Feedback) mode allows a block cipher to act like a stream cipher by processing the small increment of plain text into cipher text instead of processing it block by block. CFB mode also uses an IV to process the initial plain text.
The CTS (or Cipher Text Stealing) mode is a very versatile mode, which behaves pretty much like CBC mode. The CTS mode handles any length of plain text and produces cipher text that matches the length of the plain text.
The OFB (or Output Feedback) mode works pretty much like CFB mode; the only difference is the way that the internal buffer (shift register) is handled.
The System.Security.Cryptography Namespace
The System.Security.Cryptography namespace provides a simple way to implement security in your .NET application using the cryptography classes. Some of the cryptography classes are pure .NET managed code, and some of them are wrappers for the unmanaged Microsoft Crypto API. You can find out if the cryptography class is a managed or unmanaged code by looking at the class name. All the unmanaged providers end with the suffix CryptoServiceProvider, and all managed providers end with the Managed suffix. For example, if you look at the hashing classes such as MD5CryptoServiceProvider, SHA1Managed, and SHA256Managed, you can figure out that SHA1Managed and SHA256Managed are pure .NET managed implementations of the SHA algorithm.
The Crypto API is Microsoft's API for accessing cryptographic functions built in to the Windows platform. Microsoft recently released CAPICOM, an ActiveX wrapper around the Crypto API to simplify Crypto API programming in Visual Basic 6, but it implements only a subset of the API.
Note that the .NET Framework supports the use of strong key lengths in all encryption algorithms. However, for encryption algorithms that are implemented on top of Crypto API, you need to install a High Encryption Pack to upgrade your version of Windows:
For Windows 2000 users, Service Pack 2 includes the High Encryption Pack. It can also be obtained from the following URL: http://www.microsoft.com/windows2000/downloads/recommended/encryption/.
For Windows NT 4.0 users, Service Pack 6a includes the High Encryption Pack; this can be downloaded from: http://www.microsoft.com/ntserver/nts/downloads/recommended/SP6/allSP6.asp.
For Windows ME, Windows 98, and Windows 95 users, Internet Explorer 5.5 includes the High Encryption Pack, or you can download it from: http://www.microsoft.com/windows/ie/download/128bit/default.asp.
Cryptography Class Hierarchy
The System.Security.Cryptography namespace provides three top-level classes, SymmetricAlgorithm, AsymmetricAlgorithm, and HashAlgorithm, representing the three main areas of cryptography as shown below:
This model also brings the flexibility of extending the namespace. For example, the System.Security.Cryptography namespace doesn't currently support the Blowfish symmetric algorithm. If we wanted to add this algorithm to this namespace, all we would have to do is to derive our Blowfish algorithm class from the SymmetricAlgorithm class, and we'd get most of the standard functionality for free. The next advantage is that all the algorithm provider classes are inherited from their algorithm implementation classes. For example, the SHA1 hashing algorithm provider (the SHA1Managed class) is derived from the SHA1 hash algorithms implementation (the SHA1 class).
Hashing with .NET
The System.Security.Cryptography namespace in the .NET Framework provides several interfaces, known as Cryptographic Service Providers (CSPs), which implement a variety of hashing algorithms and make hashing simple and straightforward.
As we saw earlier, the .NET Framework implements several well-known, secure hash algorithms, including Message Digest 5 (or MD5) and Secure Hash Algorithm (or SHA). The MD5 provider generates 128-bit hash values, and the SHA provider can generate 160-bit, 256-bit, 384-bit, and 512-bit hash values. The ComputeHash() method of both the MD5 and SHA CSPs accepts a byte array or a Stream object, and returns a hash value.
The .NET Framework also implements a key-based hash, which is often used to generate digital signatures.
The HashAlgorithm Class
All the hash algorithm classes are inherited from the HashAlgorithm abstract class. The HashAlgorithm class exposes some common methods and properties that can be used across all the hashing algorithms. The following table discusses a few of them:
Class Member
Description
Hash
Returns the computed hash value in a byte array. Hash is a public property and HashValue is a protected field-both return a byte array.
HashValue
HashSize
Returns the size of the hash in bits. HashSize is a public property and HashSizeValue is a protected field-both return an integer value.
HashSizeValue
InputBlockSize
A public property that returns an integer representing the input block size in bits.
OutputBlockSize
A public property that returns an integer representing the output block size in bits.
ComputeHash()
Computes the hash value for the given input byte array or Stream. ComputeHash() is a public method that returns the output in a byte array or Stream object.
Create()
Creates an instance of the hash algorithm that is currently in use. For example, if you are using an MD5 hash algorithm then it'll create an object type of that algorithm. Create() is a static method.
TransformBlock()
Generates the hash value for a given range of the input byte array and copies the result into another byte array. TransformBlock() is a public method that returns a byte array.
TransformFinalBlock()
Generates the hash value for a given range and returns a byte array. TransformFinalBlock() is a public method that returns a byte array.
State
Returns the state of the hash computation. This property will contain a zero before the computations and a non-zero value after a successful hash computation. State is a protected field that returns an integer value representing the current state of the hash value computation.
Let's see a simple example to compute an MD5 hash. This method takes a byte array (clear text) and returns a byte array (hash value):
byte[] ComputeMD5(byte [] input)
{
MD5CryptoServiceProvider md5Provider = new MD5CryptoServiceProvider();
return md5Provider.ComputeHash(input);
}
As you can see, the MD5 provider is very easy to use. All we have to do is create a new object of the type MD5CryptoServiceProvider and pass the byte array to the ComputeHash() method, which returns a byte array. The same technique can be used for all the hash algorithms available in the .NET Framework. Let's write a simple Windows application to hash a string using multiple hash algorithms. Here is the application in action:
The implementation of the application is quite simple. We accept a string from the user and hash it using different hash algorithms. Here is the code for the Compute Hash button's Click event handler:
private void btnCompute_Click(object sender, System.EventArgs e)
{
if (txtHash.Text.Trim() != "")
{
// Generate bytes for the input string
byte[] inputData = ASCIIEncoding.ASCII.GetBytes(txtHash.Text);
// Display the hash value in textbox
txtMDS.Text = ASCIIEncoding.ASCII.GetString(new
MD5CryptoServiceProvider().ComputeHash(inputData));
txtSHAl.Text = ASCIIEncoding.ASCII.GetString(new
SHA1Managed().ComputeHash(inputData));
txtSHA256.Text = ASCIIEncoding.ASCII.GetString(new
SHA256Managed().ComputeHash(inputData));
txtSHA384.Text = ASCIIEncoding.ASCII.Getstring(new
SHA384Managed().ComputeHash(inputData));
txtSHA512.Text = ASCIIEncoding.ASCII.Getstring(new
SHA512Managed().ComputeHash(inputData));
}
}
First, we call the GetBytes() method of the ASCIIEncoding class to convert the string variable into a byte array. Then, we create a new object for each algorithm, and pass the input byte array to the ComputeHash() method. Then, we call the GetString() method of the ASCIIEncoding class again to convert the byte array into a string. Quite simple, isn't it-don't forget to add using directives for the System.Text namespace and System.Security.Cryptography namespaces to run this code.
Important If you're dealing with non-ASCII strings, you can use the UnicodeEncoding class in the System.Text namespace.
Using Hash Values for Authentication
Hashing techniques are very useful when it comes to authenticating users. We're going to see a simple authentication method for a Windows application using the MD5 algorithm. The username and the password supplied by the user will be authenticated against, for the purposes of illustration, an Access database.
Let's create an Access database called WroxDBAuth.mdb with a single table called Tbl_MA_Users. The table is going to store the user ID, e-mail address, password, first name, and last name, as shown below:
The e-mail address will be the login name for the users, and the password is stored in the database in the MD5 hash format.
This assures the users that their password can't be hacked. For example, the user's password "MyPass" is stored in the database as "{?*H!8*XJ~*N"H*S" in MD5 hash format. Let's build a simple authentication screen to authenticate users against the Access database:
Here is the code for the Login button:
private void btnLogin_Click(object sender, System.EventArgs e)
{
if (txtEmail.Text.Trim() != "" && txtPwd.Text.Trim() != "")
authenticateUser();
}
First, we check that something was entered in the username and the password text boxes. If something was entered, we call the authenticateUser() method. In the authenticateUser() method, we're connecting to the Access database and querying the table that matches the login name entered by the users.
eprivate bool authenticateUser()
{
bool bRtnValue = false;
string strConn = "PROVIDER=Microsoft.Jet.OLEDB.4.0;" +
"DATA SOURCE=c:\\DB\\WroxDBAuth.mdb;";
OleDbConnection Conn = new OleDbConnection(strConn) ;
Conn.Open();
String strSQL = "SELECT Pwd FROM Tbl_MA_Users WHERE Email = '" +
txtEmail.Text + "'";
OleDbCommand Cmd = new OleDbCommand(strSQL,Conn);
//Create a datareader, connection object
OleDbDataReader Dr = Cmd.ExecuteReader(
System.Data.CommandBehavior.CloseConnection);
//Get the first row and check the password.
if (Dr.Read())
(
Next, we pass the clear text password entered by the user into the GenerateMD5Hash() method, which returns the hashed string. If the current hashed password stored in the database and the hash generated by the GenerateMD5Hash() method are the same, we display the message Password was successful!; otherwise we displaying the message Invalid password:
if (Dr["Pwd"].ToString() == GenerateMD5Hash(txtPwd.Text))
{
MessageBox.Show(this,"Password was successful!");
bRtnValue = true;
}
else
{
MessageBox.Show(this,"Invalid password.");
}
}
else
{
MessageBox.Show(this,"Login name not found.");
}
Dr.Close();
return bRtnValue;
}
The GenerateMD5Hash() method is very simple. First, we convert the input string into a byte array using the ASCIIEncoding class. Next, we create a new object of type MD5CryptoServiceProvider and call its ComputeHash() method to generate the hash value. Then, we convert the hash value into a string and send it back to the caller:
string GenerateMD5Hash(string input)
{
// Generate bytes for the input string
byte[] inputData = ASCIIEncoding.ASCII.GetBytes(input);
// Compute the MD5 hash
MD5 md5Provider = new MD5CryptoServiceProvider();
byte[] hashResult = md5Provider.ComputeHash(inputData);
return ASCIIEncoding.ASCII.GetString(hashResult);
}
This is a simple procedure to implement, and it gives us an excellent security model for applications. In this example, we've used the MD5 algorithm. In the same way, we could use any of the other hash algorithms such as SHA1 to implement the application.
Important The only problem with this approach is that if the user wants his or her password e-mailed back to him or her, we won't be able to do it, since we can't convert the hash value back to clear text. However, we can always reset the password and send the new password back to the user.
Keyed Hash Values
The keyed hash or HMAC (Hash Message Authentication Code) algorithms are very similar to the hash algorithms, except that they generate the hash values based on a key. The HMAC algorithms are useful in the same way as the hash algorithms. For example, an HMAC value can be used to verify the integrity of a message transmitted between two parties that agree on a shared secret key. This is similar to the symmetric algorithm.
HMAC combines the original message with the key to compute a hash value. The sender computes the HMAC of the clear text and sends the HMAC with the clear text. The recipient recalculates the HMAC using the clear text and the sender's copy of the key. If the computed HMAC matches with the one sent from the other end, then the recipient knows that the original message has not been modified, since the message digest hasn't changed. In this way, the receiver can test the authenticity of the transmission. HMACs are commonly used as digital signatures.
The .NET Framework supports the HMACSHA1 and MACTripleDES algorithms. The HMACSHA1 algorithm computes keyed hash values using the SHA1 algorithm, and the MACTripleDES algorithm computes it based on the Triple-DES algorithm. We're going to see a simple example on how to use the HMAC classes. We'll build a Windows application that shows the HMAC value for the given clear text value and key.
First, we convert the input string and the key into byte arrays. Then we create an object of type HMACSHA1 and pass this object into a CryptoStream object. Then, we use the standard stream operations to read the input array and close the stream. The Hash property of the HMACSHA1 object returns the HMAC value. The same process is repeated for the MACTripleDES algorithm:
void ProcessKeyedHash(string input, string key)
{
try
{
// Generate bytes for the input string
byte[] inputData = ASCIIEncoding.ASCII.GetBytes(input);
byte[] keyBytes = new byte[16];
keyBytes = ASCIIEncoding.ASCII.GetBytes(key);
// Compute HMACSHA1
HMACSHA1 hmac = new HMACSHA1(keyBytes);
CryptoStream cs = new CryptoStream(Stream.Null, hmac,
CryptoStreamMode.Write);
cs.Write(inputData, 0, inputData.Length);
cs.Close();
txtHMACSHA1.Text = ASCII Encoding.ASCII.GetString(hmac.Hash);
// Compute the MACTripleDES
MACTripleDES macTripleDES = new MACTripleDES(keyBytes);
txtMACTripleDES.Text = ASCIIEncoding.ASCII.GetString(
macTripleDES.ComputeHash(inputData));
}
catch (Exception e)
{
MessageBox.Show(this, e.ToString());
}
}
Here is the application in action:
The only difference between these two algorithms is that the HMACSHA1 algorithm accepts keys of any size, and produces a hash value that is 20 bytes long. On the other hand, the MACTripleDES algorithm uses key lengths of 8, 16, or 24 bytes, and produces a hash value eight bytes long. If the key length is different from the requirement then an exception is thrown.
Symmetric Transformation with .NET
The System.Security.Cryptography namespace supports the DES, Triple-DES, RC2, and Rijndael symmetric algorithms. In this list, only the Rijndael algorithm is a managed implementation; the other algorithms use their counterparts in the Microsoft Crypto API.
The SymmetricAlgorithm Class
All the symmetric algorithm classes are inherited from the SymmetricAlgorithm class. The SymmetricAlgorithm class exposes some common methods and properties that can be used across all the hashing algorithms. The following table discusses a few of them:
Class Member
Description
Key
KeyValue
Specifies the secret key for the symmetric algorithm. Key is a public property and KeyValue is a protected field-both return a byte array.
KeySize
KeySizeValue
Specifies the size of the secret key in bits. KeySize is a public property and the KeySizeValue is a protected field-both return an integer value representing the length of the key in bits.
LegalKeySizes
LegalKeySizesValue
Specifies the valid key sizes in bytes for the current symmetric algorithm. LegalKeySizes is a public property and LegalKeySizesValue is a protected field-both return a KeySizes array.
IV
IVValue
Specifies the initialization vector for the symmetric algorithm. IV is a public property and IVValue is a protected field-both return a byte array.
BlockSize
BlocksizeValue
Specifies the block size in bits for the current symmetric algorithm. BlockSize is a public property and BlockSizeValue is a protected field-both return an integer.
LegalBlockSizes
LegalBlockSizesValue
Specifies the valid block size supported by the current symmetric algorithm. LegalBlockSizes is a public property and LegalBlockSizesValue is a protected field-both return a KeySizes array.
Mode
ModeValue
Specifies the mode of symmetric operation used by the current algorithm. Mode is a public property and ModeValue is a protected field-both return a CipherMode.
Padding
PaddingValue
Specifies the padding mode used by the current symmetric algorithm. Padding is a public property and PaddingValue is a protected field-both return a PaddingMode.
CreateEncryptor()
The CreateEncryptor() method creates a symmetric encryption object using the key and the initialization vector specified. CreateEncryptor() is a public method and returns an ICryptoTransform interface.
CreateDecryptor()
The CreateDecryptor() method creates a symmetric decryption object using the key and the initialization vector specified. CreateDecryptor() is a public method and returns an ICryptoTransform interface.
GenerateKey()
The GenerateKey() method generates a random key for the symmetric algorithm and overrides the value stored in the Key property. GenerateKey() is a public method and returns a random key in a byte array.
GenerateIV()
The GenerateIV() method generates a random initialization vector for the symmetric algorithm and overrides the value stored in the IV property. GenerateIV() is a public method and returns a random vector in a byte array
ValidKeySize()
Indicates whether the specified key size is valid for the current symmetric algorithm. ValidKeySizes() is a public method and returns an integer.
Let's start out by exploring the symmetric algorithms using the DES algorithm. Since symmetric algorithms tend to be faster than asymmetric ones, symmetric algorithms are good candidates for bulk encryption/decryption operations such as encrypting and decrypting entire files. We'll write a Windows application that will encrypt and decrypt files using the DES algorithm.
The user interface will provide options for locating a file using the Windows Common Dialog controls. There will be options provided to encrypt and decrypt a file with a secret key.
The Encrypt button will add a .enc extension to the source file when generating the encrypted destination file. Here is what the code looks like for the Encrypt button.
private void button1_Click(object sender, System.EventArgs e)
{
if (encryptData(encFile.Text, encFile.Text + ".enc" , txtKey.Text) == true)
MessageBox.Show(this, "Done!", "Encryption Status", MessageBoxButtons.OK,
MessageBoxIcon.Information);
else
MessageBox.Show(this, "The encryption process failed!", "Fatal Error",
MessageBoxButtons.OK, MessageBoxIcon.Stop);
}
The Encrypt button event calls the encryptData() method, passing in the source filename, destination filename, and the secret key for the encryption operation. Let's take look at the encryptData() method.
First, we create an object type of DESCryptoServiceProvider and assign the secret key supplied by the user to the Key property. Then we call the GenerateIV() method to generate an initialization vector for the encryption operation. Next, we create the DES encryption object by calling the CreateEncryptor() method of the DESCryptoServiceProvider class. After that, we instantiate two FileStream objects, one in read mode (the source file), and the other in write mode (the destination file) to encrypt the file:
// The encryptData method will encrypt the given file using the DES algorithm
public bool encryptData(string sourceFile, string destinationFile,
string cryptoKey)
{
try
{
// Create the DES Service Provider object and assign the key and vector
DESCryptoServiceProvider DESProvider = new DESCryptoServiceProvider();
DESProvider.Key = ASCIIEncoding.ASCII.GetBytes(cryptoKey);
DESProvider.GenerateIV();
ICryptoTransform DESEncrypt = DESProvider.CreateEncryptor();
// Open the source and destination file using the file stream object
FileStream inFileStream = new FileStream(sourceFile,
FileMode.Open, FileAccess.Read);
FileStream outFileStream = new FileStream(destinationFile, FileMode.Create,
FileAccess.Write);
The initialization vector (or IV) is always used to initialize the first block of plain text for encryption. We‘ve already talked about this in the Modes section.
Once we've created these objects, we need a CryptoStream object to which we write the encrypted file. We pass the DES encryption object and the output file stream into the CryptoStream's constructor. Then we read the content of the input file and write it back into the CryptoStream. Then, we close all the stream objects and return true:
// Create a CrytoStream class and write the encrypted out
CryptoStream CryptoStream = new CryptoStream(outFileStream, DESEncrypt,
CryptoStreamMode.Write);
// Declare the byte array of the length of the input file
byte[] bytearrayinput = new byte[inFileStream.Length - 1];
// Read the input file stream in to the byte array and write
//it back in the CryptoStream
inFileStream.Read(bytearrayinput, 0, bytearrayinput.Length);
CryptoStream.Write(bytearrayinput, 0, bytearrayinput.Length);
// Close the stream handlers
CryptoStream.Close();
inFileStream.Close();
outFileStream.Close();
return true;
}
catch (Exception e)
{
MessageBox.Show(this, e.ToString(), "Encryption Error",
MessageBoxButtons.OK, MessageBoxIcon.Stop);
return false;
}
}
The decryption process does the opposite of the encryption process. Let's look at the Decrypt button code. First, we retrieve the original filename from the selected one by removing the .enc extension from the filename. Then we call the decryptData() method with the source, destination, and the secret key.
private void button2_Click(object sender, System.EventArgs e)
{
string decFileHame = decFile.Text.Replace(".enc", "");
if (decryptData(decFile.Text, decFileName, txtKey.Text) == true)
MessageBox.Show(this, "Done!", "Decryption Status", MessageBoxButtons.OK,
MessageBoxIcon.Information);
else
MessageBox.Show(this, "The decryption process failed!", "Fatal Error",
MessageBoxButtons.OK, MessageBoxIcon.Stop);
}
The decryptData() method works in a very similar way to encryptData(). We create an object of type DESCryptoServiceProvider and assign the key to it. Then, we generate a new VI and create a new DES decryption object by calling the CreateDecryptor() method. Next, we read the source file into the CrytoStream and transform the content into a new file. Finally, we close the stream objects and return true:
// The decryptData method will decrypt the given file using the DES algorithm
public bool decryptData(string sourceFile, string destinationFile,
string cryptoKey)
{
try
{
// Create the DES Service Provider object and assign the key and vector
DESCryptoServiceProvider DESProvider = new DESCryptoServiceProvider();
DESProvider.Key = ASCIIEncoding.ASCII.GetBytes(cryptoKey);
DESProvider.GenerateIV();
FileStream DecryptedFile = new FileStream(sourceFile, FileMode.Open,
FileAccess.Read);
ICryptoTransform desDecrypt = DESProvider.CreateDecryptor();
CryptoStream cryptostreamDecr = new CryptoStream(DecryptedFile, desDecrypt,
CryptoStreamMode.Read);
StreamWriter DecryptedOutput = new StreamWriter(destinationFile);
DecryptedOutput.Write(new StreamReader(cryptostreamDecr).ReadToEnd());
DecryptedOutput.Flush();
DecryptedOutput.Close();
return true;
}
catch (Exception e)
{
MessageBox.Show(this, e.ToString(), "Decryption Error",
MessageBoxButtons.OK, MessageBoxIcon.Stop);
return false;
}
}
Using Other Symmetric Algorithms
Since all the symmetric algorithms are derived from the SymmetricAlgorithm class, it is very easy to implement the encryption/decryption process with the previous code base. For example, if you want to use the RC2, Triple-DES, or Rijndael algorithm, all you've to do is replace the following declaration in the encryptData() and decryptData() methods with the appropriate declarations shown below:
DES
DESCryptoServiceProvider DESProvider = new DESCryptoServiceProvider();
RC2
RC2CryptoServiceProvider RC2Provider = new RC2CryptoServiceProvider();
Triple-DES
TripleDESCryptoServiceProvider tDESProvider = new
TripleDESCryptoServiceProvider();
Rijndael
RijndaelManaged RijndaelProvider = new RijndaelManaged();
If you make this change, our file encrypter/decrypter application will work fine.
The success of the symmetric encryption and decryption process is based on the key value. If you don't supply a proper key length to the algorithm, then a CryptographicException will be raised. The key sizes supported by the algorithm can be fetched by accessing the LegalKeySizes property. This property returns an array of KeySizes objects. The KeySizes class has three integer public properties -MaxSize, MinSize, and SkipSize. The MaxSize and MinSize properties specify the maximum key size (in bits) and the minimum key size (in bits) respectively. The SkipSize returns the interval between the valid key sizes in bits.
The following table lists the key sizes supported by the major algorithms:
Algorithm
Key Size
DES
64 bits or 8 bytes
RC2
128 bits or 16 bytes
Triple-DES
192 bits or 24 bytes
Rijndael
256 bits or 32 bytes
The strength of the encryption is also based on the key. The larger the key, the better the encryption. Thus the likelihood of a hacker being able to decrypt the data with a brute-force attack is decreased. However, there is one more constraint that we should remember-the bigger the key, the more time required for the encryption and decryption process.
Asymmetric Transformation with .NET
As we discussed earlier in the chapter, asymmetric algorithms are based on the concept of public and private keys (PKI). The System.Security.Cryptography namespace supports two asymmetric algorithms: RSA and DSA.
The AsymmetricAlgorithm Class
Both the RSA and DSA algorithms inherit from the base class AsymmetricAlgorithm. The AsymmetricAlgorithm class exposes some common methods and properties that can be used across all the hashing algorithms:
Class Member
Description
KeySize
KeySizeValue
Specifies the size of the key modules in bits. KeySize is a public property and KeySizeValue is a protected field-both return an integer value representing the length of the key in bits.
LegalKeySizes
LegalKeySizesValue
Specifies a valid key size in bytes for the current asymmetric algorithm. LegalKeySizes is a public property and LegalKeySizesValue is a protected field-both return a KeySizes array.
KeyExchangeAlgorithm
Specifies the key exchange algorithm used when communicating between two ends and the way the public key and private key will be exchanged. This is a public property that returns a string representing the name of the key exchange algorithm used.
SignatureAlgorithm
Specifies the name of the algorithm used to sign the current object. This is a public property that returns a string representing the name of the signature algorithm used.
FromXmlString()
Reconstructs an Asymmetric object from an XML file. This is a public method that takes a string as input.
ToXmlString()
Returns an XML representation of the current algorithm object. This is a public method that returns a string.
As we've already discussed, many of the cryptography algorithms are implemented on top of the Crypto API library. The .NET Framework wraps the Crypto API library with sets of managed classes, called Cryptographic Service Providers (or CSPs). The CspParameters class is used to send values to and receive values from the unmanaged Crypto API.
The Cryptographic Service Providers are plug-ins for the Crypto API. These plug-ins are encryption engines that perform the encryption/decryption process.
The CSP operation is based on an enumeration value called CspProviderFlags. The CspProviderFlags enumeration supports two values: UseDefaultKeyContainer and UseMachineKeyStore. If the UseDefaultKeyContainer option is specified, the key information is read from the default key container. If the UseMachineKeyStore option is specified, the key information is read from the computer's key container.
CSPs maintain a database to store public/private key pairs. Some CSPs maintain their key container in the Registry, whereas others maintain it in other locations, such as in Smart Cards or encrypted, hidden files.
Using the RSA Algorithm
The RSA algorithm is implemented in the RSACryptoServiceProvider class, which inherits from the RSA class. The RSA class inherits from the AsymmetricAlgorithm class. The RSA algorithm allows us to encrypt, decrypt, sign data with a digital signature, and verify the signature. We'll look at these one by one in this section.
Let's start with a simple encryption/decryption approach. Since the RSA algorithm is an asymmetric algorithm, it is slower than its symmetric counterpart. Therefore, the RSA algorithm is best for small amounts of message encryption. Accordingly, we'll build a Windows application that will encrypt and decrypt the message entered by the user using PKI technology.
Whenever you create a new default constructor instance of the RSACryptoServiceProvider class, it automatically creates a new set of public/private key information, ready to use. We can also store the PKI values into XML files. This demonstration will be our first example. Let's build a UI as shown below.
This UI allows us to see the clear text as well as the encrypted cipher text. Our code will also show the public key and the private key used in this process and we'll have an option to store the public and private keys in different XML files.
Since we're going to use the RSA auto generated public/private keys, we declare a class level static object.
static RSACryptoServiceProvider objRSAProvider;
Here is the code for the Encrypt button:
if (txtClearText.Text.Trim() != "")
{
// Initialize the RSA Cryptography Service Provider (CSP)
rsaProvider = new RSACryptoServiceProvider();
UTFSEncoding utf8 = new UTFSEncoding();
byte[] clearText = utf8.GetBytes(txtClearText.Text.Trim());
// Encrypting the data received
txtCipherText.Text = Convert.ToBase64String(rsaProvider.Encrypt(clearText,
false));
// Show the private key
txtPrvKey.Text = rsaProvider.ToXmlString{true);
// Show the public Key
txtPubKey.Text = rsaProvider.ToXmlString(false);
}
We create a new object type of RSACryptoServiceProvider. Then we transform the user-entered message into a byte array using the UTF8Encoding class. Then we call the Encrypt() method of the RSACryptoServiceProvider class, pass the input byte array and a second parameter of false.
The second parameter in the Encrypt() method deals with the mode of operation. If you are running Windows 2000 OS with SP2 or higher then you can set this parameter to true, which will use the OAEPpadding method. When set to false it'll use PKCS version 1.5.
The Encrypt() method returns an encrypted byte array. We're using the ToBase64String() method of the Convert class to convert the byte array into a string for display in the textbox. We also display the private/public key generated by the RSA algorithm in two text boxes using the ToXmlString() method of the RSACryptoServiceProvider class. ToXmlString() takes a Boolean as input and if the value is false, it generates the public key value as an XML string; if the value is true, it generates the private key.
Here is the code for our Decrypt button. This method just calls the Decrypt() method of the RSACryptoServiceProvider object:
private void btnDecrypt_Click(object sender, Systern.EventArgs e)
{
if (txtCipherText.Text.Trim() ! = "")
{
//Convert the input string into a byte array
byte[] bCipherText =
Convert.FromBase64String(txtCipherText.Text.Trim());
//Decrypt the data and convert it back to a string
string strValue =
ASCIIEncoding.ASCII.GetString(objRSAProvider.Decrypt(
bCipherText, false));
//Display the decrypted string in a MessageBox
MessageBox.Show(this, strValue, "Decrypted value",
MessageBoxButtons.OK,MessageBoxIcon.Information);
}
}
Now let's look at saving the public/private key into an XML file. We display the SaveFile common dialog box to get the desired filename from the user. Then we load the XML data into an XmlDocument object and call its Save() method to save the data to disk:
private void btnSave_Click(object sender, System.EventArgs e)
{
SaveFileDialog saveFileDialog1 = new SaveFileDialog();
saveFileDialogl.Filter = "XML files (*.xml)|*.xml|All files(*.*)|*.*" ;
saveFileDialog1.FilterIndex = 2 ;
saveFileDialog1.RestoreDirectory = true ;
if(saveFileDialog1.ShowDialog() == DialogResult.OK)
{
// Write the content to an XML file
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.LoadXml(this.txtPubKey.Text);
// Save the document to a file.
xmlDoc.Save(saveFileDialog1.OpenFile());
}
}
Here is the application in action:
As you can see, the clear text, encrypted cipher text, and the decrypted clear text is shown on the screen with the public and private key information.
Loading the Public and Private Keys
In the previous example, we saw how to encrypt and decrypt data using the auto-generated public and private keys. We also saw how to save the keys into an XML file. However, if you want to reuse previously created or saved keys, you can do this by initializing the class with a populated CspParameters object. Let's see an example of this. Create a UI like the previous one, but this time we'll load the public and private keys from the XML files:
Here is the code to load a key from XML. First, we display an OpenFile dialog to get the filename from the user. Then, we load the file into an XmlTextReader object and display the key information in the appropriate text box:
private void btnLoadPub_Click(object sender, Systern.EventArgs e)
{
// Show the open file dialog
openFileDialog1.Title = "Select the Public Key file";
openFileDialog1.Filter = "XML Files (*.xml)|*.xml";
if(openFileDialog1.ShowDialog() == DialogResult.OK)
{
string fileName = openFileDialog1.FileName;
btnEncryptl.Enabled = true;
// Load the document
Xml Text Reader xmlReader = new XmlTextReader(fileName);
xmlReader.WhitespaceHandling = WhitespaceHandling.None;
xmlReader.Read();
// Assign the public key to the textbox
txtPubKeyl.Text = xmlReader.ReadOuterXml();
}
}
Now let's look at the code behind the Encrypt button. First, we create an object of the CspParameters class and set the flag to use the machine store to look for the PKI keys. Then we give the key container the name WroxRSAStore.
private void btnEncryptl_Click(object sender, System.EventArgs e)
{
if (txtClearText1.Text.Trim() != "")
{
try
{
CspParameters cspParam = new CspParameters();
csp Param.Flags = CspProviderFlags.UseMachineKeyStore;
cspParam.KeyContainerName = "WroxRSAStore";
cspParam.ProviderName = "MS Strong Cryptographic Provider";
// CryptoAPI constant -> PROV_RSA_FULL = 1
// This provider type supports both digital
// signatures and data encryption, and is considered
// general purpose. The RSA public-key algorithm
// is used for all public-key operations.
cspParam.ProviderType = 1;
Once we've initialized the parameters for the CSP, we create a new object of type RSACryptoServiceProvider using our CspParameters object. Then, we assign the public key by calling the FromXml String() method. After that, we perform our usual process of converting the string into a byte array and passing the byte array into the Encrypt() method and converting the byte array back to a string:
// Initializing the RSA Cryptography Service Provider (CSP)
RSACryptoServiceProvider rsaProvider1 = new
RSACryptoServiceProvider(cspParam);
// Load the public key
rsaProvider1.FromXmlString(txtPubKey1.Text);
UTF8Encoding utf8 = new UTF8Encoding();
byte[] clearText = utf8.GetBytes(txtClearText1.Text);
// Convert encrypted text to base64
txtCipherText1.Text = Convert.ToBase64String(
rsaProvider1.Encrypt(clearText, false));
}
catch (Exception e)
{
MessageBox.Show(this, e.ToString());
}
}
}
The decryption method is again very simple-we just create an RSA CSP object and assign the private key to it. We then decrypt the message using the Decrypt() method:
private void btnDecrypt1_Click(object sender, System.EventArgs e)
{
if (txtClearText1.Text.Trim() != "")
{
// Initialize the RSA Cryptography Service Provider (CSP)
RSACryptoServiceProvider rsaProvider1 = new RSACryptoServiceProvider();
// Load the private key
rsaProvider1.FromXmlString(this.txtPriKey1.Text);
// Encrypt the data received
MessageBox.Show(this, ASCIIEncoding.ASCII.GetString(
rsaProvider1.Decrypt(Convert.FromBase64String(
txtCipherText1.Text.Trim()), false)), "Decrypted value",
MessageBoxButtons.OK, MessageBoxIcon.Information);
}
}
Here is the application in action:
There is one limitation of the RSA algorithm that you should know-the Encrypt() method can only encrypt up to 16 bytes if the High Encryption pack is installed. Otherwise, it can only encrypt 5 bytes.
Reading an X509 Certificate
A certificate is like a 'voucher' that contains information about the person holding the voucher, such as who authorized the certificate, its public keys, and its expiration information. Certificates are signed by a certifying authority (or CA), such as VeriSign or Thawte. The Microsoft Certificate Server also allows us to create 'self-signed' certificates. However, they may not be trusted by the rest of the world, since this is not issued by a well-known CA, but they are very useful in an intranet scenario.
Server certificates are used to identify the trustworthiness of the server, and client certificates to identify a client to the server. A CA issues both client and server certificates after verifying the identity. For example, when the client is requesting a web resource, it can also send a client certificate along with the request. The server can then determine who the client is and authorize or deny them.
Thawte provides the client certificate free of charge but VeriSign charges a fee for it.
Client certificates are usually installed on web clients such as browsers and e-mail clients. You can view all the client certificates installed in IE by clicking on Tools | Internet Options… and selecting the Content tab in the dialog box. Click on the Certificates… button here. As you can see, | have two client certificates installed in IE 6:
The Cryptography namespace also contains the child namespace X509Certificates. This contains just three classes used to represent and manage Authenticode X509 v.3 certificates. The X509Certificate class exposes the static methods CreateFromCertFile() and CreateFromSignedFile() to create an instance of the certificate.
The CreateFromCertFile() method reads the content of the X509 certificate from a certificate file and the CreateFromSignedFile() method reads the content of the X509 certificate from a Digitally Signed file. We will use the CreateFromCertFile() method to read the contents of the X509 client certificate:
private void btnView_Click(object sender, System.EventArgs e)
{
// Read the client certificate from the file
// into the object variable of the type X509Certificate
X509Certificate clientCert =
X509Certificate.CreateFromCertFile("C:\\test.cer");
StringBuilder sb = new StringBuilder();
sb.Append("Issuer Name: " + clientCert.GetIssuerName() + "\n");
sb.Append("Public Key String: " + clientCert.GetPublicKeyString() + "\n");
sb.Append{"Key Algorithm: " + clientCert.GetKeyAlgorithm().ToString() + "\n");
sb.Append("Serial Number: " + clientCert.GetSerialNumberString() + "\n");
sb.Append("Effective Date: " +
clientCert.GetEffectiveDateString().ToString() + "\n");
sb.Append("Expiration Date: " +
clientCert.GetExpirationDateString().ToString() + "\n");
MessageBox.Show(this, sb.ToString());
}
We've declared an object of type X509Certificate, and we've used the CreateFromCertFile() method to read the certificate content into the object. Then we've read the main properties of the certificate and disolaved it to a message box:
Cryptography and Network Programming
So far in this chapter we've learnt all about cryptography-now it's time to use some of the techniques in network programming. Remember the simple UDP chat utility that we wrote in Chapter 6? The UDP chat application is very simple. It takes the local and remote port numbers and the IP address and passes the information back and forth.
Here is the code before adding any cryptography algorithms. The Send() method here gets the datagram, converts it into a byte array, and sends the data using the Send() method of the UdpClient class. The Receiver() method here also works in the same way.
private static void Send(string datagram)
{
// Create UdpClient
UdpClient sender = new UdpClient();
// Create IPEndPoint with details of remote host
IPEndPoint endPoint = new IPEndPoint(remoteIPAddress, remotePort);
try
{
// Convert data to byte array
byte[] bytes = Encoding.ASCII.GetBytes(datagram);
// Send data
sender.Send(bytes, bytes.Length, endPoint);
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
finally
{
// Close connection
sender.Close();
}
}
public static void Receiver()
{
// Create a UdpClient for reading incoming data.
UdpClient receivingUdpClient = new UdpClient(localPort);
// IPEndPoint with remote host information
IPEndPoint RemoteIpEndPoint = null;
try
{
Console.WriteLine(
"-----------*******Ready for chat!!!*******-----------");
while(true)
{
// Wait for datagram
byte[] receiveBytes = receivingUdpClient.Receive(
ref RemoteIpEndPoint);
// Convert and display data
string returnData = Encoding.ASCII.GetString(receiveBytes);
Console.WriteLine("−" + returnData.ToString());
}
}
catch (Exception e)
{
Console.WriteLine(e.ToString ());
}
}
Let's use the Rijndael symmetric algorithm to send secure information between two chat clients. The Rijndael algorithm is a symmetric algorithm and it is best for bulk data transfer. This is the main reason that I've chosen this algorithm. We're going to add encryptData() and decryptData() methods to this class to take care of the secure communication process. When we're sending information to the remote socket we'll call the encryptData() method to encrypt the data, and then we'll pass the encrypted byte array to the other end. When we receive the encrypted byte array on the other end, we'll call the decryptData() method to decrypt the byte array.
As you can see, our design is very simple. To support the symmetric algorithms, we've added two private class level members that store the shared key and vector.
private static IPAddress remoteIPAddress;
private static int remotePort;
private static int localPort;
private static UTF8Encoding Utf8Encod;
private static string CryptoKey = "!i~6ox1i@]t2K'y$";
private static string CryptoVT = "!~x7Oq{6+q1@#VI$";
The encryptData() method takes a string as input and returns a byte array.
static byte[] encryptData(string theDataGram)
{
byte[] bCipherText = null;
try
{
We create a new object type of RijndaelManaged, and assign the shared key and vector. Then we create an ICryptoTransform object using the CreateEncryptor() method.
// Create the Rijndael Service Provider object and assign the
// key and vector to it
RijndaelManaged Rijndael Provider = new RijndaelManaged();
Rijndael Provider.Key = Utf8Encod.GetBytes(CryptoKey);
RijndaelProvider.IV = Utf8Encod.GetBytes(CryptoVI);
ICryptoTransform RijndaelEncrypt = Rijndael Provider.CreateEncryptor();
Now, we convert the datagram into a byte array using the UTF8Encoding class. Then we declare a MemoryStream object and use a CryptoStream object to perform the cryptographic transformation- you may recall that we had a fleeting look at the CryptoStream class in Chapter 2.
// Convert string to byte array
byte[] bClearText = Utf 8Encod.GetBytes{theDataGram);
MemoryStream Mstm = new MemoryStream();
//Create Crypto Stream that transforms a stream using the encryption
CryptoStream Cstm = new CryptoStream(Mstm, RijndaelEncrypt,
CryptoStreamMode.Write);
//Write out encrypted content into MemoryStream
Cstm.Write (bClearText, 0, bClearText.Length);
Ctms.FlushFinalBlock();
We create the byte array back from the MemoryStream and return the byte array to the caller.
//Get the output
bCipherText = Mstm.ToArray();
//Close the stream handlers
Cstm.Close();
Mstm.Close();
}
catch (Exception e)
{
Console.WriteLine(e.ToString ());
}
return bCipherText;
}
The decryptData() method, which does the decryption process, is similar to the encryptData() method. The decryptData() method takes a byte array as input and returns a string as output.
static string decryptData(byte[] bCipherText)
{
string sEncoded ="";
try
{
// Create the RijndaelManaged Service Provider object and assign
// the key and vector to it
RijndaelManaged RijndaelProvider = new RijndaelManaged();
RijndaelProvider.Key = Utf8Encod.GetBytes(strCryptoRey);
RijndaelProvider.IV = Utf8Encod.GetBytes(strCryptoVI);
ICryptoTransform RijndaelDecrypt= RijndaelProvider.CreateDecryptor();
//Create a MemoryStream with the input
MemoryStream Mstm = new MemoryStream(bCipherText, 0, bCipherText.Length);
//Create Crypto Stream that transforms a stream using the decryption
CryptoStream Cstm = new CryptoStream(Mstm, RijndaelDecrypt,
CryptoStreamMode.Read);
// read out the result from the Crypto stream
StreamReader Sr = new StreamReader(Cstm);
sEncoded = Sr.ReadToEnd();
Sr.Close();
Cstm.Close();
Mstm.Close();
}
catch (Exception e)
{
Console.WriteLine(e.ToString ());
}
return sEncoded;
}
Let's call the encryptData() and decryptData() methods in the proper places in the chat application. The Send() method calls the encryptData method, with the string entered by the user, and sends the result via the Send() method of the UdpClient. The Receiver() method passes the received byte array into the decryptData() method and displays the decrypted message.
private static void Send(string datagram)
{
...
try
{
// Convert string to byte array
//byte[] bClearText = Utf8Encod.GetBytes(datagram);
// Encrypting the data recived
byte[] bytes = encryptData(datagram);
// Send data
sender.Send(bytes, bytes.Length, endPoint);
}
...
}
public static void Receiver()
{
...
while(true)
{
// Wait for datagram
byte[] receiveBytes = receivingUdpClient.Receive(ref RemoteIpEndPoint);
//Decrypt the incoming byte array
string returnData = decryptData(receiveBytes);
Console.WriteLine("−" + returnData.ToString());
}
...
}
We can also implement this with an asymmetric algorithm such as RSA-let's see an example of this. We'll just rewrite the encryptData() and decryptData() methods using the RSA algorithm. The rest of the implementation will remain the same.
Before writing the encryptData() and decryptData() methods we've to store the public key and the private key in private class level variables.
private static IPAddress remoteIPAddress;
private static int remotePort;
private static int localPort;
private static UTF8Encoding Utf8Encod;
private static string PubKey =
"
e8g/yby624K85qJLwMMzwCru7b+kNTA2dYaK4Nk+FkZMLCVmomiW1zns2KsT1aF9hwr32Nyje3OuJDlHqB
tcOpCGbo+kJ+JC88BM1J9AkdoAa+SE=
private static string PriKey =
"
e8g/yby624K85qJLwMMzwCru7b+kNTA2dYaK4Nk+FkZMLCVmomiW1zns2KsT1aF9hwr32Nyje3OuJ01HqB
tcOpCGbo+kJ+JC88BM1J9AkdoAa+SE=
3BoisxTvnh8Xt
g/02fTGtr/k8OXUOiEfKwAKzWje36v8zkTfIc4EzdZbRskJywq1NMo9U1EHM3DUv+Ya/KGPzQ==
0AcCph/CdQeB2/M+q3BSlzimr9Chw9zaHk1x8MBCHdRB9c26VcS0AmKW+G4Vz jWJjI6cK8j/GQjhnRn7Ub
BypQ==
GwcmBIAHa1hez2yJTQ==
guEeFYjp2RIatLMrcA4QV4KV3+DzQWaeQ==
yhKD7J5zLGIzz+71C5QjVi2dRwtvjGJaexOTi+TRIv2fT/LhWmsCDQ==
1gXY8AznfnLC05eXrDIuo/YBsY2qredFDQaLqWIZiiq4ur7kWoFHakAbHCGeC3p2+bmLyrYr2nm8OgjOc1
NUneE8ASoKWfnbcWxW377Oeogj16frPUoAgwU1gFURdTxozgNLThVtNItrc3Doa5eJ+U7pRSz2edE=
Here is how the encryptData() and decryptData() methods look. In the encryptData() method we create a new CspParameters object, and use the object to create a new RSA algorithm object. Next, we assign the public key from the private class member. Then, we call the Encrypt() method of the RSA object.
static byte[] encryptData(string strDataGram)
{
CspParameters theCspParam = new CspParameters();
theCspParam.Flags = CspProviderFlags.UseMachineKeyStore;
theCspParam.KeyContainerName = "WroxRSAStore";
// Initializing the RSA Cryptography Service Provider (CSP)
RSACryptoServiceProvider theRSAProvider = new
RSACryptoServiceProvider(objCspParam);
//Set the Load the public key
theRSAProvider.FromXmlString(PubKey);
// Convert string to byte array
byte[] bClearText = Utf8Encod.GetBytes(strDataGram);
// Encrypting the data received
byte[] bytes = theRSAProvider.Encrypt(bClearText, false);
theRSAProvider.Clear();
return bytes;
}
The decryptData() method does pretty much the same thing. It creates a new RSA object and assigns the private key from the class member. Then it calls the Decrypt() method and sends the string back to the caller.
static string decryptData(byte[] bCipherText)
{
// Initializing the RSA Cryptography Service Provider (CSP)
RSACryptoServiceProvider theRSAProvider = new RSACryptoServiceProvider();
//Load the private key
theRSAProvider.FromXmlString(PriKey);
// Encrypting the data received
string strRtnData = Utf8Encod.GetString(theRSAProvider.Decrypt(bCipherText,
false));
theRSAProvider.Clear();
return strRtnData;
}
This implementation of the encryptData() and decryptData() methods can be replaced with the previous example and the code will work fine. The only thing that we've to be careful about in this example is the size limitations of the Encrypt() method-we talked about these limitations earlier.
E-Mail Protocols
E-Mail Protocols
Back in Chapter 1, we touched on the various protocols that exist for sending and receiving e-mails across the network. E-mail protocols are broad in the sense that they don't only pertain to .NET. Ever since the rise of Internet messaging, e-mail has been a cornerstone of electronic communication. I'm sure almost everyone reading this book has had some prior experience sending e-mails within a programming language, so we aren't really covering some new and exciting technology that .NET brings to the table. What is exciting is how easy and seamless .NET makes network programming, especially with regard to using the various e-mail protocols to send and retrieve e-mails, and to perform various tasks that may have been convoluted or difficult in the past. In this chapter, we are going to give a high-level overview of the various e-mail protocols and how they are accessed and used in a .NET environment. Particular examples will be given in C#, and discussion will follow.
E-mail in a Nutshell
It's quite easy to think of e-mail like ordinary mail that you may send out. The big difference is that no stamps are needed! Well, that and the fact that e-mail sending is almost instantaneous, and the postal service is usually very slow. There are various advantages to sending e-mail, though. E-mail can be sent to many recipients at once, as well as forwarded on and on very easily. However, in both cases we know that there is a starting point and an endpoint to a communication session. There is also a delivery mechanism that transfers the mail from one point to another along its mail route until it reaches its final destination. In the most generic sense, there are two components interacting in this scenario:
Message Transfer Agent (MTA)
User Agent (E-mail client)
In our real-world snail mail example, the Transfer Agent may be a postal or carrier company such as the USPS (United States Postal Service), FedEx, or UPS. The sender and receiver are quite obvious. In the electronic world, the place of the Transfer Agent is taken by the implementation of the SMTP (Simple Mail Transfer Protocol), which is responsible for directing e-mail from the starting point (sender) to the endpoint (receiver), bouncing the e-mail across various machines on the Internet. Keep in mind that this method stems from roots in TCP/IP, and is the most prevalent messaging topology used today. However, there are a number of other systems and standards that have been used for messaging in the past.
How does E-Mail Work?
When we think about how e-mails are actually sent across the Internet, we must first realize that the Internet is nothing more than hundreds of thousands of computers that are networked together, using the same standard protocols for communication. How does that fit into our e-mail situation? Endpoints are simply mailboxes or message stores on e-mail servers (although you could also view the sender and receivers as the starting and endpoints, and the message stores as holding places for the messages until they are received by a client program). A mail server is simply the machine (or virtual machine) that handles sending and receiving messages, and communicates with other machines in the process of handling e-mails.
A common scenario in a company may be to have employees using a Microsoft Outlook mail client that connects to the company's mail server (which may be running Exchange). All e-mail messages that come in and go out of that company are handled by the mail server. It acts as both the endpoint and starting point for any e-mails sent out over the Internet:
This is a simplified situation, as many companies have multiple servers to handle e-mail communications.
Notice that the mail clients (such as Outlook) communicate with the mail server using SMTP, and that the mail server also communicates with other machines over the Internet using SMTP. POP3 (Post Office Protocol) and IMAP (Internet Message Access Protocol) are also depicted in the diagram; these protocols only allow mail clients to access and retrieve e-mail. To send mail, e-mail clients use SMTP.
E-Mail Protocols
As this scenario shows, the three main topics for e-mail programming are SMTP, POP3, and IMAP. This is not to say that they are the only topics, just the ones that are most important to us developers when we are writing our applications that send and retrieve e-mail messages. This is because these are the standard protocols for sending and retrieving e-mail. We must keep in mind that these aren't the only ones, but the most commonly used and accepted methods. Other messaging standards such as X.400 exist as an alternative to TCP/IP related messaging. For more information about the X.400 protocol, see http://www.itu.int/ and http://www.alvestrand.no/x400/.
Internet mail is largely defined by a number of standards and recommendations made by companies and individuals close to Internet technology research and design. These standards are endorsed by the IMC or Internet Mail Consortium (http://www.imc.org/) and the IETF or Internet Engineering Task Force (http://www.ietf.org). The current set of Internet mail standards is composed of many related RFCs (Request For Comments), recommendations, and statements on common practices. Not all standards are fully endorsed or full IETF standards documents, but are considered stable and are used throughout the industry for developing e-mail software.
There are many RFCs available on the IMC web site (http://www.imc.org/rfcs.html). These documents are detailed and speak in much greater depth about every aspect of e-mail handling. For our purposes we are going to touch on a few of these RFCs within our discussion of various topics in this chapter. In particular, we will look at RFC 2821, which defines the SMTP protocol, and RFC 2822, which defines what an e-mail message should look like. For more information about these RFCs, take a look at the IMC web site.
SMTP
Simple Mail Transfer Protocol, or SMTP, is outlined in RFC 2821, and defines the interaction between mail servers that are transporting e-mails. It's good to note that SMTP for the most part uses TCP as a transport protocol (see RFC 1090 for SMTP over X.25). In essence, an SMTP session consists of a conversation between two machines that are trying to hand off or pass along an e-mail message. A simplified view of this is shown in the following figure:
As you can see, an SMTP session can be summed up in a simple communication between two computers in which the e-mail is relayed along. At this point, the e-mail will have bounced around the Internet to finally reach Machine 2 that is the mail handler for i-netway.com. If for instance, there was no user at this e-mail address (the address was invalid) or the mail handler didn't have any way to deliver the message, the machine would have replied that the e-mail was undeliverable.
Another View of an SMTP Session
Another way to look at an SMTP e-mail session would be to take a look at a Telnet connection to an actual e-mail server. Most ISPs disable this sort of access, as it allows for hackers and the like to easily spoof e-mails, which is not a good thing! This sample is only meant to show an example of the conversation between an SMTP e-mail server and a client. If someone at Wrox were trying to send me an e-mail via Telnet, the conversation would look something like this:
open i-netway.com 25
Trying. . . Connected to i-netway.com
220 I-NETWAY.COM - Server ESMTP (PMDF v4.3-10 #2381)
helo wrox.com
250 I-NETWAY.COM OK, WROX.COM.
mail from:
250 Address Ok.
rcpt to:
250 krowczyk@i-netway.com OK.
data
354 Enter mail, end with a single ".".
SUBJECT:E-mail Chapter
Andy, thanks for the chapter!
.
250 OK.
//by quitting, the message is sent
quit
221 Bye received. Goodbye.
See for a brief discussion of using the Microsoft Telnet client.
As you can see, the session begins with the Telnet command to open a connection to the i-netway.com e-mail server. This is followed by a few different SMTP commands that are needed to send the e-mail message, and the responses from the SMTP server. We looked at the most common SMTP commands in Chapter 5, where we built a simple SMTP client, but now let's look at the complete list.
SMTP Command Summary
Here is a quick summary of the various SMTP commands that are available. For more detailed explanations, please read RFC 2821.
Command Summary
Description
HELLO (HELO)
Identifies the SMTP client to the SMTP server.
MAIL (MAIL)
Initiates a mail transaction to deliver an e-mail to one or more mailboxes.
RECIPIENT (RCPT)
Identifies the recipient to whom the mail data is to be sent. If the data is to be sent to more than one recipient, multiple RCPT commands can be used.
DATA (DATA)
Marks the start of the mail data. The data following the DATA command is appended to the mail buffer. The mail data may contain any of the 128 ASCII character codes. The end of the data is marked by the sequence..
SEND (SEND)
Initiates a mail transaction to deliver an e-mail to one or more terminals. Here it is important to note that ‘terminal' refers to a user's terminal screen. Because most clients are not terminal clients nowadays, the SEND, SOML, and SAML commands are for the most part obsolete. They are included here for the sake of completeness, because they are part of the RFC.
SEND or MAIL (SOML)
Initiates a mail transaction to deliver an e-mail to one or more terminals or mailboxes. The e-mail is delivered to the terminal of each recipient if that recipient is active on the host and accepting terminal messages, and to every recipient's mailbox.
(SAML)
Initiates a mail transaction to deliver an e-mail to one or more terminals or mailboxes. The e-mail is delivered to the terminal of each recipient if that recipient is active on the host and accepting terminal messages, and to every recipient's mailbox.
RESET (RSET)
Aborts the current mail transaction. All data will be discarded, and all buffers cleared. The receiver must send an OK reply.
VERIFY (VRFY)
Asks the receiver to confirm that the following argument is a valid username. If it is, the full name of the user (if known) and the fully specified mailbox are returned.
EXPAND (EXPN)
Asks the receiver to confirm that the following argument is a mailing list, and if so, to return the names of the members of that list. The full names of the users (if known) and the fully specified mailboxes are returned in a multi-line reply. This command is similar to the VRFY command, but is used for multiple recipients.
HELP (HELP)
Asks the receiver to send helpful information to the sender. The command can take an argument (such as a command name) and return more specific information as a response.
NOOP (NOOP)
This command has no effect, except that the receiver should send an OK reply. No operation takes place, and no other commands or data should be affected.
QUIT (QUIT)
On receiving a QUIT command, the receiver must send an OK reply, and then close the connection to the sender.
TURN (TURN)
On receiving a TURN command, the receiver must either (1) send an OK reply and take on the role of the SMTP client; or (2) send a refusal reply and retain the role of the SMTP server. This command is deprecated because of security concerns. Poor authentication implementations would allow a client machine to take on the role of the SMTP server and divert e-mail messages.
Reply Codes
If you look back at the example of sending an e-mail using SMTP through Telnet, you will see that the server sends back a reply code for each SMTP command. Actually, the reply codes sent back by the server provide us with much information about the status of the current e-mail transaction. Let's take a look at a typical reply code and see what information it contains:
Since the first two digits of the SMTP reply code are the most important to us, let's look at what they indicate. Please note that the first digit only gives the status of the command. The subsequent digits give us greater detail as to what the success or error was caused by.
First Digit
indicates preliminary acceptance of the command, pending confirmation
indicates successful completion of the command
indicates intermediate acceptance of the command, pending further information
indicates temporary negative status
indicates failure
Second Digit
syntax
connection
mail
To see how this works, let's go over a few of the most common reply codes that we might receive back from an SMTP transaction call. The error or status messages given may differ from SMTP server to SMTP server, but the reply code means the same thing in each case. The status message is really just a human-readable description of what went wrong.
Reply Code
Message
500
Syntax error, command unrecognized
501
Syntax error in parameters or arguments
502
Command not implemented
503
Bad sequence of commands
220
Service ready
221
Service closing transmission channel
421
Service not available, closing transmission channel
250
Requested mail action OK, completed
354
Start mail input, end with.
550
Requested action not taken, mailbox unavailable
553
Requested action not taken, mailbox name not allowed
554
Transaction failed
RFC 2821 defines all of the reply codes available, and the numeric codes are definitive. This means that you can't define new codes as you see fit! However, since we are talking about an agreed standard protocol, the response codes should be predefined and set in stone.
A Typical E-Mail Message
Before we start talking about accessing e-mails with POP3 and IMAP, let's first briefly touch on the content and structure of a mail message as it relates to the other important standards document that we spoke about earlier. RFC 2822 describes the content of what an e-mail message should look like. Think of it as sort of a 'schema' or outline of a typical mail message. The following is a typical mail message:
Received: from MAILCLUSTER [111.111.111.111] by mail.brinkster.com with ESMTP
(SMTPD32-6.05) id AF1F18B600FA; Fri, 28 Jun 2002 05:40:47 −0400
Received: by MAILCLUSTER with Internet Mail Service (5.5.2653.19)
id; Fri, 28 Jun 2002 10:39:30 +0100
Message-ID:
From: Wrox Press
To: "'Andrew Krowczyk'"
Subject: C# Networking Chapter
Date: Fri, 28 Jun 2002 10:39:28 +0100
Importance: high
X-Priority: 1
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain
X-RCPT-TO:
X-UIDL: 323073316
Status: U
Andy, thanks for the work you've done!
-Wrox Press
This typical e-mail structure consists of both message headers and message text. Some information is required for the message to conform to the e-mail standards while some information is optional and can be included or excluded depending on many factors.
Required Information
There are a few headers that are generally required in all e-mails:
FROM: The agent (person, system, or process) that created the message. This should be a single authenticated machine address generated by the sending agent. As you would imagine, this tells us who/what is sending us the e-mail message.
DATE: The date the message was sent. The only optional parts of the date specification are the day of the week and the seconds. The timezone may be given in usual denotations such as CST, EDT, GMT. The timezone is preferred as a numeric offset from the GMT. In the message above, you will see +0100 to denote the GMT offset.
One Recipient Address: At least one recipient address must be used, which can be either To, Cc, or Bcc.
Some Optional Information
There are also other headers and information that may be optional and not required in every message. A few of these are listed below:
REPLY-TO: The reply-to header is often used to designate the preferred e-mail address for responses to be sent to. This is often used by list mail and other processes to correctly identify the return e-mail address location.
SENDER: Why would the sender be optional? This is really the same thing as the required FROM field listed above. This field is intended for use when the sender of the e-mail is not the author of the e-mail, or is one of a group of authors. This shouldn't be used if it's identical to the FROM field. The SENDER field must be present if it's different from the FROM field.
An example of the SENDER field usage would be.
FROM: "Joe Someone"
SENDER: INFO-SAMPLE Discussion
TO: Multiple recipients of list INFO-SAMPLE Discussion
Received Headers
One of the most important parts of an e-mail message is the data that is transmitted in the received headers of the mail message. This provides debugging information, as well as a good look at where the e-mail came from and how it got from point A to point B.
Our sample above contains the following received lines:
Received: from MAILCLUSTER [111.111.111.111] by mail.brinkster.com with ESMTP
(SMTPD32-6.05) id AF1F18B600FA; Fri, 28 Jun 2002 05:40:47 −0400
Received: by MAILCLUSTER with Internet Mail Service (5.5.2653.19)
id; Fri, 28 Jun 2002 10:39:30 +0100
The received lines tell us many things:
They are the postmaster's primary debugging tool. By reading the header, we can find out much information about where the mail came from and how it got to the final destination.
They tell which systems have touched or tampered with the mail.
Each MTA (Mail Transfer Agent) that relays a message attaches its own Received header line.
RFC 2882 requires that MTAs add their own received line when they handle the mail, and they are prohibited by the RFC from touching the received lines put in by other mailers.
The received headers show us the path, hop by hop, that the mail took from the sender to the receiver.
An example of how the messages can be traced via the received lines in the headers is shown in the diagram below:
Viewing Headers in Outlook
Since the headers contain so much information, it's worth seeing quickly how to view header information from Microsoft Outlook. First, right-click on the message that you wish to view the header information for, and select Options. The Internet headers for the message are displayed at the bottom of the resulting dialog:
Note that Internet headers might not be displayed for e-mails that are sent internally on the same network.
What About MIME?
Now that we've seen the basics of an e-mail message and looked at a high-level overview of the e-mail protocol, we come to the question, 'What about attachments?' E-mail was originally a text-only messaging system using just ASCII characters. But that quickly changed into the need to 'attach' binary files of different types to the e-mail messages and have the attachments transferred along with the ASCII text messages. So, to address this need, MIME (Multi-purpose Internet Mail Extensions) was developed; MIME defines extensions to the SMTP protocol to support binary attachments of arbitrary format.
When we look at what MIME is composed of, we find that it really encompasses two main functions:
MIME encodes binary data so that it can be passed over the Internet. There are two points to notice here:
We must remember that the Internet is a 7-bit ASCII world.
8-bit extensions don't work, as there are issues with line length and file formatting.
MIME attaches a label or tag to the encoded data so that the content can be determined and interpreted at the end point of the message. For example, this is a movie file, or a Microsoft Excel document.
MIME uses a new encoding scheme that is called BASE64. It also adds new SMTP headers that describe the attached document. The idea is actually quite simple-we are just encoding the binary data into a different bit data representation and then piggybacking that data with the e-mail itself as an attached bit of data. Once the endpoint e-mail client receives the message, the new headers tell it that there is an attached document (or attached documents) embedded within the e-mail message, and the client can properly decode and display the document. The RFCs that define MIME and its composition are RFC 2045 through RFC 2049.
MIME Headers
When we take a look at the headers that must be used when MIME is used to attach a bit of information we will find that there are a few different fields that are available for use. Let's look first at the required field:
Required Fields
Description
MIME-Version
This field indicates which version of MIME is being used (currently 1.0).
Now the optional fields:
Optional Fields
Description
Content-type
This field describes what format this part of the message is in, such as text, message, application, multipart, image, audio, among others. The default type is ASCII text.
A few common content types include:
text/plain
text/html
application/binary
application/postscript
image/gif
image/jpeg
Content-transfer_encoding
This header tells how to decode the message. There are a few different encoding schemes that may be used.
Base64. This encoding is used to encode binary data in 7-bit ASCII data.
7-bit. No encoding, case insensitive.
8-bit. No encoding.
binary. No encoding.
x-token proprietary encoding designation. The x designates that this is a non-standard status encoding scheme. This simply means that the content is not encoded with a standardized coding scheme, which in most cases is bad, as it doesn't fall into interoperability guidelines.
Content-ID
ID field that allows one body to make reference to another. Similar to Message-ID (unique identifier in the e-mail message structure). Generally this field is optional.
Content-description
A textual description about the encoded data. For example, an image content type might contain 'a picture of the moon'.
Content-disposition
Field used to tell the e-mail client if the content should be displayed 'inline' with the message or as an 'attachment' to the message. There is a parameter called Filename that is a suggestion for a name if the encoded data is detached from the mail message.
Now let's take a quick look at a typical mail message that includes a MIME attachment:
From: krowczyk@i-netway.com (Andrew Krowczyk)
Subject: Sample message with Word Document Attachment
To: mailto:editor@wrox.com
MIME-version: 1.0
Content-type: MULTIPART/MIXED; BOUNDARY="Boundary_[ID_nf991kyavAuSo/HeKKQ]"-
Boundary_[ID_nf991kyavAuSo/HeKKQ]
Content-type: text/plain; charset=us-ascii
Hi, please see the attached Word Document.
-Andy-Boundary_[ID_nf991kyavAuSo/HeKKQ]
Date: Fri, 3 Jul 2002 16:43:34 −0700
Content-type: application/mac-binhex40; name=sample_worddoc.doc
Content-disposition: attachment; filename=sample_worddoc.doc
PGh0bWw+DQo8aGVhZD4NCjx0aXRsZT6q967mpcC/y7hgrKGwyjwvdG10bGU+DQo8bWV0YSBodHRw
LWVxdWl2PSJDb250ZW50LVR5cGUiIGNvbnRlbnQ9InRleHQvaHRtbDsgY2hhcnNldDliaWclIj4N
CjxsaW5rIHJlBD0ic3R5bGVzaGVldCIgaHJlZj0iaHR0cFDovL3d3dy5raW5nLmNvbS50dy9wbS5j
c3MiIHR5cGU9InRleHQvY3NzIj4NCjwvaGVhZD4NCjxzY3JpcHQgbGFuZ3VhZ2U9IkphdmFTY3Jp
cHQiPg0KZnVuY3Rpb24gZ290b3VybChkaXl0eXBlKSB7DQogdmFyIHB3PXdpbmRvdy5vcGVuKCdo
As you can see, the MIME sections and headers define the message boundaries and the binary content. For brevity, I've left out the actual binary encoding of the document. It would look like a bunch of garbled data in ASCII characters.
For the most part, I've covered only a few topics on MIME attachments. This was only to give a brief high-level overview of how attachments are included in SMTP e-mail messages. Later in the chapter, we'll look at an example of sending e-mails that include attachments from a .NET application. Hopefully, this overview of MIME will help you to understand what our .NET code is causing to happen behind the scenes when we send e-mail messages. Now we've seen what happens when we send an e-mail, we'll look at the protocols and methods for retrieving e-mail messages.
Retrieving Client-Server E-Mail
When we think about and look at how e-mail messages are retrieved from mail server message stores, we generally think of it as a client-server relationship. We use an e-mail client, such as Microsoft Outlook, Lotus Notes, or other such mail packages to retrieve messages from a central mail server mailbox. There are also web based e-mail services such as Hotmail that follow along the same lines. There are generally three models of e-mail handling:
Offline (POP3 model)
Client connects to the mail server and pulls or retrieves e-mail down to the client.
In this instance, all the mail is stored on the client machine, not the server. Usually e-mail is deleted from the server once it's retrieved, although some mail clients give the ability to 'leave messages on the server'.
Online (Original IMAP model)
The client application connects to the server for every transaction.
Everything is stored on the server.
Disconnected (Later IMAP model)
Client and server share the storage of messages.
The server is always correct, and the client application must 'synchronize' the list of messages with the server.
One of the most important points to note is that POP and IMAP only get the mail messages from the server's mailbox. As with all e-mail transactions, sending mail requires the use of SMTP. In the next few sections, we'll talk in a little more depth about the POP and IMAP protocols for retrieving e-mails. After that, we'll delve into the code required in .NET to perform all the actions we've discussed so far.
POP3
There are actually a few versions of the POP protocol that have been around. The POP2 and POP3 standards are not even compatible from a protocol standpoint. But since the huge majority of POP clients are POP3, we'll restrict our discussion to that protocol.
So what does POP3 allow us to do? As we've already learned, POP3 only allows a client to retrieve mail from a mail server. This protocol allows us to write nice-looking GUI (Graphical User Interface) applications that will present e-mail in a way that's easy to read and manage. For reference, POP3 is outlined in further detail in RFC 1939.
In the POP3 world, the mail exists on the mail server when it's received by the system. It just sits there, waiting to be picked up by a mail client. Once we connect to the mail server with a client mail program, the POP3 client copies the mail from the server to the local machine's hard drive. This method locks us into reading the mail on the client machine. Why would this make a difference? Depending on the situation, POP3 access might not be ideal. For instance, suppose we have a mail account that we read both at work and at home. Let's suppose that we have Microsoft Outlook installed on both the office and home machines. If we connect at the office and Outlook downloads four new mail messages, those messages will generally be removed from the server and transported to the local hard drive on the office machine. This means that we've lost access to those messages from our home machine. Because POP3 moves the messages from the server to your client machine, sharing a POP3 e-mail system between multiple machines can be troublesome.
POP3 Via Telnet
The only reason that I'm going to cover this is to quickly show the steps or procedures that allow us to access a typical POP3 mailbox on a mail server. The internal methods used in the .NET C# System.Web.Mail namespace would do something very similar to this while retrieving messages using the POP3 protocol. Although most of this is abstracted away by the .NET runtime components, it's interesting to see what happens behind the scenes.
open mail.someserver.com 110
Trying. . . Connected to MAIL.SOMESERVER.COM
+OK test.someserver.COM MultiNet POP3 Server Process v4.0(1) at Fri 20-Jun-2002
3:21PM-CST
user krowczyk //designate user
+OK User Name (krowczyk) ok. Password, please.
pass thisismypassword //enter password
+OK 3 messages in folder INBOX (V4.0)
list 2 //list gives message size
+OK 2 7124 //in bytes
stat //stat gives total message
+OK 3 14749 //size in bytes
quit
+OK POP# MultiNet test.somewhere.COM Server exiting (3 INBOX messages left)
Connection closed by Foreign Host
POP Commands
A typical POP session consists of a client connecting to the POP3 server on TCP port 110 (the default). Once the client is connected, the POP3 server sends back a connection-greeting message that acknowledges the connection and starts the POP3 session. Commands are then issued and responded to by the client and server until the connection has been closed and the session ends.
According to the RFC, POP3 commands consist of a keyword optionally followed by arguments. Some of the specifics for commands are as follows:
Commands are terminated by a CRLF sequence
Keywords are separated by a SPACE character
Keywords are three or four characters long
Each argument may be up to 40 characters in length
Responses may be up to 512 characters in length
+OK designates a positive response
-ERR designates a negative response or error condition
Required Commands
Description
USER [name]
Username sent to the server.
PASS [password]
The password.
QUIT
Terminates or ends the current session.
DELE [msg]
Deletes mail from the server.
RSET
Undoes any changes made during the current session.
STAT
Returns the number of messages on the server.
RETR [msg]
Retrieves the content of a message.
LIST [msg]
Returns information about the message in parameter. Such as size in bytes. If no parameter is given, a list with all messages and their sizes is returned.
NOOP
Does nothing but cause the server to respond with a positive response.
Optional Commands
Description
TOP [msg] [n]
The server sends the headers of the message, the blank line separating the headers from the body, and then the number of lines of the indicated message's body, [msg] is the message number desired, [n] specifies the top n lines to be retrieved.
UIDL [msg]
If an argument is given, the server issues a positive response with a line containing information about the specified message. Called a 'unique id listing' for the message selected. The unique-id of a message is an arbitrary server-determined string, consisting of one to 70 characters in the range 0×21 to 0×7E, which uniquely identifies a message within a maildrop.
APOP [mailbox] [digest]
A string identifying a mailbox and an MD5 digest string. The MD5 algorithm takes a message of arbitrary length as an input, and produces a 128-bit fingerprint message digest of the input. Typically used within RSA cryptography and such.
The POP3 protocol gives us basic functionality for retrieving messages from our mail server. It serves its purpose very well, and is pretty much the standard that the majority of mail servers and clients use.
IMAP
IMAP gives us everything that POP3 doesn't. It provides capability for on-line, off-line, or disconnected modes of operation. It allows us to control folders on both the client machine as well as the server. It also provides enhanced authentication (POP3 authentication is relatively weak). And it allows the use of multiple mail servers that can work together with the same mail client.
Let's take a brief look at how an IMAP client and server might interact:
IMAP4 is outlined and documented in RFC 2060. Some of IMAP's features include:
Server-side Mailbox Manipulation
Ability to add messages to a remote mailbox
Support for and notification of simultaneous updates in shared mailboxes
New mail notification
Multiple Mailbox Support
Remote folder management: list/create/delete/rename
Support for folder hierarchies
Access to message types other than e-mail, such as NetNews
Online Performance Optimization
Provision for determining message structure without downloading the entire message
Selected retrieval of MIME body parts
Server-based search and retrieval to minimize data transfer
Perfect for Roaming Power Users
Shared folders overcome limitations of POP3 access on more than one machine
Performance optimization helps with slow mail server connections
Although IMAP gives us much more advanced functionality than POP3, you'll find that many of the e-mail users and clients in the world are using POP3 servers for mail retrieval. IMAP is often much more complex to implement than POP3. For offline processing mode, POP3 and IMAP have almost equivalent possibilities, but for online or disconnected processing mode, IMAP is clearly superior. However, because of POP3's inherent simplicity, we'll slant our discussion in this chapter to cover the POP3 implementation in more detail.
.NET and E-Mails
Now that we've had an overview of mail protocols, we can get to the most interesting bits of information. Once you understand the way the SMTP, POP3, and IMAP protocols work, the code being presented here should be very easy to follow.
We'll start by looking at how to send e-mails using the .NET System.Web.Mail namespace, which allows us to send e-mail and attachments using SMTP. We'll also develop some sample applications that implement the SMTP and NNTP (the Network News Transport Protocol, which is used for newsgroup access and is very similar to SMTP).
SMTP
In your past life (when you programmed without .NET), you may have used CDONTS to send e-mails within your applications. This was commonly done from within both ASP and Visual Basic. There were often third-party ActiveX components that took the place of CDONTS and made sending e-mails a bit easier than dealing with Microsoft's confusing objects. In .NET, Microsoft has wrapped all the SMTP e-mail functionality up and rolled it directly into the .NET Framework under the System.Web.Mail namespace. Using the classes within this namespace, we can easily construct and send mail, with attachments if desired, using the SMTP service built into US.
Using the System.Web.Mail namespace actually allows us to send messages using the CDOSYS (Collaboration Data Objects for Windows 2000) message component. In fact, .NET simply wraps the functionality of the underlying messaging components. This allows the e-mail messages to be delivered either through the SMTP mail service built into Windows 2000 or through an arbitrary SMTP server.
It's been my experience in the past that third-party components often provide the deep and managed functionality that my applications have needed. In particular, if you need a component that contains more functionality than the System.Web.Mail namespace, you can take a look at the following components:
EasyMail.NET (http://www.quicksoft.com). This also provides a very nice parsing library and IMAP capabilities.
aspNetEmail (http://www.aspnetemail.com)
Smtp.NET (http://www.exclamationsoft.com/exclamationsoft/smtp.net/default.asp)
System.Web.Mail Namespace
The System.Web.Mail namespace contains three classes and three enumerations:
Class
Description
Mail Attachment
Represents the attachments of an e-mail.
MailMessage
Represents the e-mail message itself.
SmtpMail
Responsible for sending a MailMessage via SMTP.
Enumeration
Description
Mail Encoding
Specifies the encoding of the message-Base64 or UUEncode.
MailFormat
Specifies the format of the message. Either HTML or Text.
MailPriority
Specifies the priority of the message. High, Medium, or Low
Sending a message using this namespace requires us first to construct a MailMessage object to represent the e-mail. Once we've set the properties of this object and populated it with the details of the e-mail message, we send it using the SmtpMail class
You are most likely to use these various properties when initializing the MailMessage object:
MailMessage Class Properties
Description
Attachments
The list of attachments (MailAttachment objects) that are transmitted with the e-mail.
Bcc
A semicolon-delimited list of e-mail addresses that receive a Blind Carbon Copy (Bcc) copy of the e-mail.
Body
The body of the e-mail.
BodyFormat
Specifies the MailFormat of the e-mail. This can be either MailFormat.Text or MailFormat.Html.
Cc
A semicolon-delimited list of e-mail addresses that receive a Carbon Copy (Cc) of the e-mail.
From
The e-mail address of the sender.
Priority
Specifies the MailPriority of the e-mail.
Subject
The subject of the e-mail.
To
The e-mail address of the recipient.
UrlContentBase
Gets or sets the Content-Base HTTP header. The URL base of all relative URLs used in an HTML body.
UrlContentLocation
Gets or sets the Content-Location HTTP header for the e-mail message.
Headers
Read-only property that consists of the list of custom headers to be transmitted with the message.
Constructing a MailMessage Object
The first step in constructing a MailMessage object is to set a reference to the System.Web.dll assembly, since the System.Web and System.Web.Mail namespaces aren't available until the reference is set:
After we have added the reference to System.Web.dll, we can include it in a using statement, and instantiate our MailMessage object:
using System.Web.Mail;
// Create a mail message object
MailMessage email = new MailMessage();
Next, we'll set up a few properties of the e-mail:
// Set message parameters
email.From = "someone@someone.com";
emai1.To = "krowezyk@i-netway.com";
email.Subject = "Test message using SmtpMail";
email.BodyFormat = MailFormat.Text;
email.Body = "This is just a test message";
Adding an Attachment
Adding an attachment is as easy as creating one or more MailAttachment objects and adding them to the Attachments list of the MailMessage object. Here's a quick look at the members of the MailAttachment class:
MailAttachment Class Properties
Description
Encoding
Type of encoding for the e-mail attachment. This can be either MailEncoding.Base64 or MailEncoding.UUEncode.
Filename
Name of the attachment file.
Now let's add an attachment. We do this by calling the Add() method of the Attachments collection, passing in the path of the file that we want to attach to the mail, and the MailEncoding to use:
email.Attachments.Add(new MailAttachment(@"c:\testfile.txt",
MailEncoding.Base64));
We can add any number of attachments to the e-mail message in this way by continuing to add Mail Attachment objects to the Attachments list of the MailMessage.
An SmtpMail Object
To send an e-mail message that you've composed, you need to call a method of the SmtpMail object. The interesting thing to note is that there is only one really important method of the SmtpMail class- the static Send() method. This method sends off the mail message to the SMTP server. It can take a MailMessage object as a parameter, or it can take four strings that specify the sender, recipient, subject, and body of the e-mail.
The SmtpMail class also has one public property:
SmtpMail Class Properties
Description
SmtpServer
This is the server to which the SMTP mail should be sent. If this property is not set, the SmtpMail object defaults to sending the mail to the Windows SMTP service for delivery.
Please note that you must have the SMTP service installed on your machine before this code will work when not specifying an SMTP server. To install the US SMTP service, select Control Panel | Add Remove Programs | Add Remove Windows Components | Internet Information Services | SMTP Service.
// Send the e-mail using the SmtpMail object
SmtpMail.Send(email);
//or also like this - without a mail message object
string from = "from@somewhere.com";
string to = "to@somewhere.com";
string subject = "test";
string body = "test message";
SmtpMail.Send (from, to, subject, body);
If we don't set the SmtpServer property, the e-mail will be sent via the Windows SMTP service. To set the SmtpServer property we can just use the following code:
// For example, here we want to set the SMTP server field before sending the mail
// message
SmtpMail.SmtpServer = "mail.testserver.com";
// Now send message as we did before
SmtpMail.Send(email);
The previous code snippets have shown the various parts of sending e-mail using the SMTP capabilities of .NET. Now we'll look at a sample application that ties it all together.
SMTP Mailer Application
Our finished sample SMTP mail application will be a simple .NET Windows Form application that allows us to set various properties of an e-mail and send it using a designated SMTP server. The following screenshot depicts the end result of the code we're going to cover. The form has five single-line text boxes, where the user can enter the name of the SMTP server to use, the e-mail addresses of the sender and recipient, the subject of the mail, and the path to any attachment to send with the mail. There is also a multi-line text box, where the body of the mail can be typed:
This code is available in the chapter download materials. There is really no magic to writing this sort of simple application. We basically just create a form with appropriate text boxes and enter some code in the Click event handler for the Send button that uses the SMTP mail capabilities we've been looking at to send the mail on its way.
Important Code
As before, we need to add a reference to the System.Web.dll assembly, and a using directive to point to the System.Web.Mail namespace. Apart from the standard Windows code (generated by Visual Studio .NET), we only need to add the code behind the Send button. This is very similar to the code we've already seen:
private void button1_Click(object sender, System.EventArgs e)
{
// Create mail message
MailMessage email = new MailMessage();
// Set message parameters
email.From = txtFrom.Text;
email.To = txtRecipient.Text;
email.Subject = txt Subject.Text;
email.BodyFormat = System.Web.Mail.MailFormat.Text;
email.Body = txtMessage.Text;
// Add attachment
email.Attachments.Add(new MailAttachment(txtAttachment.Text,
MailEncoding.Base64))
// Set SMTP server
SmtpMail.SmtpServer = txtSmtpServer.Text;
// Now send message
SmtpMail.Send(email);
}
Check out the download materials for the full sample application.
POP3
If you've had any experience with .NET mail programming or development, you will probably know that .NET doesn't provide inherent support for retrieving mail from POP3 and IMAP mailboxes. Although we've already covered the SMTP component of the .NET Framework that merrily allows us to send SMTP e-mails and so on, what we haven't said yet is that the System.Web.Mail namespace is basically just a wrapper for the CDOSYS component. This unmanaged component is really used underneath to send the e-mails.
So how do we retrieve mails using .NET? We could try using CDO to retrieve mails, but that will only work for Microsoft Exchange Servers, and not all the mail servers out there in the world are based on Microsoft Exchange! What we really want to do, seeing that this is a networking book, is to code our own POP3 e-mail class that uses the POP3 protocol to retrieve e-mails. We'll use some of the networking namespaces that exist in .NET to accomplish this goal.
Creating a POP3 C# Class
Our first task in creating a class that handles POP3 messages is to determine what exactly a message consists of. First create a new C# Windows Application project called POPMail.
Let's start off by creating a class that represents the core of a POP3 e-mail message, so add a new class to the project called POP3EmailMessage:
public class POP3EmailMessage
{
// Define public members
public long msgNumber;
public long msgSize;
public bool msgReceived;
public string msgContent;
}
This class contains a message number, the size of the message (in bytes), a flag indicating whether or not the message has been received from the server, and a string containing the actual content of the e-mail message. If you remember in our protocol discussion, these fields are common within a typical e-mail message and are used/shown when retrieving a message. For example, we need an index (msgNumber) to the message that we want to deal with when listing out or retrieving from the POP3 server.
System.Net.Sockets.TcpClient
As we've seen in the last few chapters, the .NET Framework presents us with a rich base of classes to deal with low-level networking needs. In our case, we want to harness the TcpClient class, because we will make remote connections to the mail server using the TCP protocol. We will therefore create a class that is derived from the System.Net.Sockets.TcpClient namespace:
// Define the POP3 class
public class POP3 : System.Net.Sockets.TcpClient
{
Connecting to a Server
There are a few things that this class must be able to do, the first of which is connect to the POP3 server passing in a username and password.
Remember our Telnet session back in our discussion about the POP3 protocol to draw similarities to what we are doing here.
So, let's start by writing a ConnectPOP() method. This takes three parameters-the name of the server to connect to, and the username and password of the mailbox we want to access. Note that it will call some methods that are presented later in the chapter:
public void ConnectPOP(string sServerName, string sUserName, string sPassword)
{
// Message and the server resulting response
string sMessage;
string sResult;
// Call the connect method of the TcpClient class
Connect(sServerName, 110);
// Remember default port for server is 110
// Get result back
sResult = Response();
// Check response to make sure it's +OK
if (sResult.Substring(0,3) != "+OK")
throw new POPException(sResult);
// Got past connect, send username
sMessage = "USER " + sUserName + "\r\n";
// Write() sends data to the Tcp Connection
Write(sMessage);
sResult = Response();
// Check response
if (sResult.Substring(0,3) != "+QK")
{
throw new POPException(sResult);
}
// Now follow up with sending password in same manner
sMessage = "PASS " + sPassword + "\r\n";
Write(sMessage);
sResult = Response();
if (sResult.Substring(0,3) != "+OK")
throw new PQPException(sResult);
}
I've put comments in the above code to help with its readability. We connect to the POP3 server and send the username and password using the Connect() method of the TcpClient class, and our methods Write() and Response(). As we saw in the POP3 Telnet session that we discussed earlier, the POP3 server should send back a +OK response if we were successful, and a -ERR message when a failure occurs. If this happens, we throw an exception using the message sent back from the server.
When You Connect, You Must Disconnect
Since we're talking about connecting, we may as well also show the disconnect method now. To disconnect from the server, we simply need to issue the QUIT command. That makes our DisconnectPOP() method really quite simple:
public void DisconnectPOP()
{
string sMessage;
string sResult;
sMessage = "QUIT\r\n";
Write(sMessage);
sResult = Response();
if (sResult.Substring(0,3) != "+OK")
throw new POPException(sResult);
}
So far, we have a way to connect to the POP3 server, as well as a way to disconnect from it. The typical POP3 session will consist of a call to ConnectPOP(), commands to get e-mails and so on, and then a call to DisconnectPOP().
Getting a List of Messages
We can get a list of messages on the POP3 inbox simply by issuing a LIST command to the server. The ListMessages() method shows how we can do this.
Important Note that using the ArrayList requires us to import the System.Collections namespace.
public ArrayList ListMessages()
{
// Same sort of thing as in ConnectPOP and DisconnectPOP
string sMessage;
string sResult;
ArrayList returnValue = new ArrayList();
sMessage = "LIST\r\n";
Write(sMessage);
sResult = Response();
if (sResult.Substring(0, 3) != "+OK")
throw new POPException (sResult);
while (true)
{
sResult = Response();
if (sResult == ".\r\n")
{
return returnValue;
}
else
{
POP3EmailMessage oMailMessage = new POP3EmailMessage()
// Define a separator
char[] sep = { ' '};
// Use the split method to break out array of data
string[] values = sResult.Split(sep);
// Put data into oMailMessage object
oMailMessage.msgNumber = Int32.Parse(values[0]);
oMailMessage.msgSize = Int32.Parse(values[1]);
oMailMessage.msgReceived = false;
returnValue.Add(oMailMessage);
continue;
}
}
}
As we saw in the Telnet example, sending a LIST command to the POP3 server will cause the server to send back multiple lines of text. Each line represents an e-mail message that contains a message number and a number of bytes that the e-mail message contains. It's key to note that this method only returns an array of message objects that have very minimal data in them. The way that we receive more information and the message content is by creating a mirror of the RETR command that retrieves the actual message.
Retrieving a Specific Message
To retrieve a full message from the POP3 server, we need to issue the RETR command with the msgNumber of the message that we want to retrieve. This follows the same procedure as we described before:
public POP3EmailMessage RetrieveMessage(POP3EmailMessage msgRETR)
{
string sMessage;
string sResult;
// Create new instance of object and set new values
POP3EmailMessage oMailMessage = new POP3EmailMessage ();
oMailMessage.msgSize = msgRETR.msgSize;
oMailMessage.msgNumber = msgRETR.msgNumber;
// Call the RETR command to get the appropriate message
sMessage = "RETR " + msgRETR.msgNumber + "\r\n";
Write(sMessage);
sResult = Response();
if (sResult.Substring(0, 3) != "+OK")
throw new POPException (sResult);
// Set the received flag equal to true since we got the message
oMailMessage.msgReceived = true;
// Now loop to get the message text until we hit the "." end point
while (true)
{
sResult = Response();
if (sResult == ".\r\n")
break;
else
oMailMessage.msgContent = sResult;
}
return oMailMessage;
}
Deleting a Message
Since the POP3 protocol states that a message isn't deleted from the server when the message is retrieved, we need to call the DELE command explicitly to remove the message from the server. Writing that method is also quite simple:
public void DeleteMessage(POP3EmailMessage msgDELE)
{
string sMessage;
string sResult;
sMessage = "DELE " + msgDELE.msgNumber + "\r\n";
Write(sMessage);
sResult = Response();
if (sResult.Substring(0, 3) != "+OK")
throw new POPException(sResult);
}
The Write() Method
The write method takes a message as input and writes it out to the TCP network stream. This will in effect send our command to the POP3 server we are connected to. Since C# string data types cannot be directly buffered to the network stream, we must use the ASCIIEncoding class of the System.Text namespace to get the byte representation of the string data. Once this is done, we just write that out to the network stream:
private void Write(string sMessage)
{
// Used for Data Encoding
System.Text.ASCIIEncoding oEncodedData = new System.Text.ASCIIEncoding() ;
// Now grab the message into a buffer for sending to the TCP network stream
byte[] WriteBuffer = new byte[1024];
WriteBuffer = oEncodedData.GetBytes(sMessage) ;
// Take the buffer and output it to the TCP stream
NetworkStream NetStream = GetStream();
NetStream.Write(WriteBuffer, 0, WriteBuffer.Length);
}
The Response() Method
The flip side of the Write() method described above is the Response() method. This method allows us to read data back from the POP3 server connection. This is used throughout the class to grab the result codes sent back from the server in response to commands that we've sent with each Write(). Again, we need to use the ASCIIEncoding class to get the string representation of the bytes being received across the network stream.
This code sample should be quite self-explanatory:
private string Response()
{
System.Text.ASCIIEncoding oEncodedData = new System.Text.ASCIIEncoding();
byte []ServerBuffer = new Byte[1024];
NetworkStream NetStream = GetStream();
int count = 0;
// Here we read from the server network stream and place data into
// the buffer (to later decode and return)
while (true)
{
byte []buff = new Byte[2];
int bytes = NetStream.Read( buff, 0, 1 );
if (bytes == 1)
{
ServerBuffer [count] = buff[0];
count++;
if (buff[0] == '\n')
{
break;
}
}
else
{
break;
}
}
// Return the decoded ASCII string value
string ReturnValue = oEncodedData.GetString(ServerBuffer, 0, count )
return ReturnValue;
}
PopException Class
This class simply encapsulates an application exception that we may throw in our code:
namespace POPMailException
{
public class POPException : System.ApplicationException
{
public POPException(string str) : base(str)
{
}
}
}
A Quick Console Application Sample
The full sample application given here can be found in this book's download from www.wrox.com. The sample application adds a GUI front end to the POP3 class that we just wrote, but the basic use of this class can be seen by taking a look at the Main() method of the console application listed below. This method steps through the items needed to instantiate the class and retrieve the message numbers and message text:
static void Main(string[] args)
{
try
{
POP3 oPOP = new POP3();
oPOP.ConnectPOP("mail.someserver.com", "username", "password");
ArrayList MessageList = oPOP.ListMessages();
foreach (POP3EmailMessage POPMsg in MessageList)
{
POP3EmailMessage POPMsgContent = oPOP.RetrieveMessage(POPMsg);
System.Console.WriteLine("Message {0}: {1}",
POPMsgContent.msgNumber, POPMsgContent.msgContent);
}
oPOP.DisconnectPOP();
}
catch (POPException e)
}
System.Console.WriteLine(e.ToString());
}
catch ( System.Exception e)
{
System.Console.WriteLine(e.ToString());
}
}
}
The image below is a screenshot of the sample PopMail application that I've written. It uses the POP3 class above to list the e-mails in my mailbox. The user can then retrieve the message text by inserting the number of the message to display. The entire message is then retrieved from the mailbox:
Expanding on this Class
There are many ways in which this class could be improved and expanded upon. You may want to expand the POP3EmailMessage class to include much more functionality. Individual properties that describe the headers, subject, message, and so on. I made the class simple to use and tried to follow the same sort of design as you would see via logging into a Telnet session with the POP3 server, as this stays more true to the design of the POP3 protocol and is useful for our understanding in a networking frame of mind.
There are also many popular third-party components that encapsulate this functionality very nicely. But what fun would buying one of those be?
NNTP
NNTP is commonly used to access the content of newsgroups on the Internet. The protocol is defined in RFC 977, and has been around for a while. We'll briefly cover a class that encapsulates access to NNTP newsgroups and servers, which is based upon and quite similar to the POP3 class that we described earlier. Why is it similar? Firstly, it's a basic TCP network stream-based protocol, much like SMTP and other e-mail related protocols. We haven't really touched much on NNTP up until this point in a protocol discussion, since this chapter is really targeted towards the topic of .NET and e-mail access. Therefore, if you'd like to read further information about the NNTP protocol, I'd advise reading RFC 977 located at http://www.ietf.org/rfc/rfc0977.txt?number=977 for further information. For ease of reading the code, I will list the common commands and responses that may be used against an NNTP server. These are very similar to the POP3 items listed in our previous examples.
NNTP Commands
The most common commands issued to an NNTP newsgroup server are listed in the following table:
NNTP Commands
Description
ARTICLE
Displays the header, a blank line, then the body text of the current or specified article.
GROUP
Will return the article numbers of the first and last articles in the group, and an estimate of the number of articles on file in the group.
LAST
The internally maintained 'current article pointer' is set to the previous article in the current newsgroup.
LIST
Returns a list of valid newsgroups and associated information.
NEWSGROUPS
A list of newsgroups created since a specified date and time will be listed in a similar format to the LIST command.
NEWNEWS
A list of message IDs of articles posted or received in the specified newsgroup since the date specified.
NEXT
The internally maintained 'current article pointer' is advanced to the next article in the current newsgroup. If no more articles remain in the current group, an error message is returned and the article remains selected.
POST
If posting is allowed, the article is posted to the server.
QUIT
Closes the connection with the server.
NNTP Responses
The most common responses received back from an NNTP newsgroup server are listed below. These are not specific messages, but a listing of the meaning of different digits that compose the NNTP response. For more complete response listings, please view the RFC.
NNTP Responses
Description
1xx
Informative message
2xx
Command OK
3xx
Command OK so far, send the rest of it
4xx
Command was correct, but couldn't be performed for some reason
5xx
Command unimplemented, or incorrect, or a serious program error occurred
x0x
Connection, setup, and miscellaneous messages
x1x
Newsgroup selection
x2x
Article selection
x3x
Distribution functions
x4x
Posting
x8x
Nonstandard (private implementation) extensions
x9x
Debugging output
Some common response codes include:
Common Messages
Description
100
Help text
190 through 199
Debug output
200
Server ready-posting allowed
201
Server ready-no posting allowed
400
Service discontinued
500
Command not recognized
501
Command syntax error
502
Access restriction or permission denied
503
Program fault-command not performed
You'll see more specific responses and commands as we discuss the NNTP code class below.
Creating an NNTP Class In C#
This explanation of the class will follow the same sort of presentation as the POP3 class. We will derive the NNTP class from the same System.Net.Sockets.TcpClient class. First we import the necessary namespaces:
using System;
using System.Net.Sockets;
using NNTPServerException; // Our own exception class implementation
using System.Collections;
Inheriting from the TCP Client Class
As in the POP3 client, inheriting from the TcpClient gives us a wealth of functionality that we don't need to implement ourselves, such as the network transfer layers responsible for connecting to the server and allowing us to send data across the network stream.
public class NNTP : System.Net.Sockets.TcpClient
Connecting to the Server
Our ConnectNNTP() method simply calls the TcpClient's Connect() method, passing in the server name as well as the standard port number (119) for the NNTP server connection:
public void ConnectNNTP(string sServer)
{
string sResult;
// Connect to the server on the default port #119
Connect(sServer, 119);
sResult = Response();
// In this case, a response code of 200 is an OK response
if (sResult.Substring(0, 3) != "200")
throw new NNTPException(sResult);
}
Disconnecting from the Server
Again, if we write a connect function, we must write a disconnect function. This simply sends the QUIT command to the NNTP server:
public void DisconnectNNTP()
{
string sMessage;
string sResult;
// Send the QUIT command
sMessage = "QUIT\r\n";
Write(sMessage);
sResult = Response();
// We expect a code of 205 acknowledging the quit
if (sResult.Substring(0, 3) != "205")
throw new NNTPException (sResult);
}
Getting the Newsgroups
As you might imagine, calling GetNewsGroupListing() in our class will use the LIST command to return all of the groups that are available on the NNTP server. This may be a lot, depending on the server!
public ArrayList GetNewsGroupListing()
{
string sMessage;
string sResult;
// Create an array for the return values
ArrayList ReturnValue = new ArrayList();
sMessage = "LIST\r\n";
Write(sMessage);
// Check the response, if OK continue
sResult = Response();
if (sResult.Substring(0, 3) 1= "215")
throw new NNTPException(sResult);
// While there are more results, loop and append to output array list
while (true)
{
sResult = Response();
if (sResult == ".\r\n" || sResult == ".\n")
{
return ReturnValue;
}
else
{
char[] separator = { ' '};
string[] values = sResult.Split(separator);
ReturnValue.Add(values[0]);
continue;
}
}
}
Getting News from a Group
To get the news from a specific newsgroup that is listed on the NNTP server, we simply need to create a method that calls the GROUP command, passing in the name of the newsgroup for which we want to retrieve the messages:
public ArrayList GetNews(string sNewsGroup)
{
string sMessage;
string sResult;
ArrayList ReturnValue = new ArrayList();
sMessage = "GROUP " + sNewsGroup + "\r\n";
// Write the message to the server
Write(sMessage);
sResult = Response();
// Check for successful operation
if (sResult.Substring(0, 3) != "211")
throw new NNTPException(sResult);
char[] separator = { ' ' };
string[] values = sResult.Split(separator);
// For beginning and end
long begin = Int32.Parse(values[2]);
long end = Int32.Parse(values[3]);
if (begin + 100 < end && end > 100)
begin = end - 100;
for (long i = begin; i
sMessage = "ARTICLE " + i + "\r\n";
Write(sMessage);
sResult = Response();
if (sResult.Substring( 0, 3) == "423")
continue;
if (sResult.Substring( 0, 3) != "220")
throw new NNTPException(sResult);
string sArticle = "";
while (true)
{
sResult = Response();
if (sResult == ".\r\n")
break;
if (sResult == ".\n")
break;
if (sArticle. Length < 1024)
sArticle += sResult;
}
ReturnValue.Add(sArticle);
}
return ReturnValue;
}
Posting to a Group
Posting to a newsgroup is also an easy task. It consists of simply calling the POST command with the name of the newsgroup, followed by the headers and the body of the message we want to post:
public void PostMessage(string sNewsGroup, string sSubject, string sFrom,
string sContent)
{
string sMessage;
string sResult;
sMessage = "POST " + sNewsGroup + "\r\n";
Write(sMessage);
sResult = Response();
if (sResult.Substring( 0, 3) != "340")
throw new NNTPException(sResult);
// Build message
sMessage = "From:" + sFrom + "\r\n"
+ "Newsgroups: " + sNewsGroup + "\r\n"
+ "Subject: " + sSubject + "\r\n\r\n"
+ sContent + "\r\n.\r\n";
Write(sMessage);
sResult = Response();
if (sResult.Substring( 0, 3) != "240")
throw new NNTPException(sResult);
}
The Write() Method
Again, due to the encoding/decoding that must take place to allow the C# string type to be transported in bytes over the network stream, we need to come up with our own method that writes the data in a memory buffer to the server:
private void Write(string sMessage)
{
System.Text.ASCII Encoding oEncode = new System.Text.ASCIIEncoding() ;
byte[] WriteBuffer = new byte[1024];
WriteBuffer = oEncode.GetBytes(sMessage);
NetworkStream oNetworkStream = GetStream();
oNetworkStream.Write(WriteBuffer, 0, WriteBuffer.Length);
}
The Response() Method
We also need to transform the data coming back from the server to an appropriate string format for our class representation:
private string Response ()
{
System.Text.ASCIIEncoding oEncode = new System.Text.ASCIIEncoding();
byte []ServerBuffer = new byte[1024];
NetworkStream oNetworkStream = GetStream();
int count=0;
while (true)
{
byte []LocalBuffer = new Byte[2];
int bytes = oNetworkStream.Read(LocalBuffer, 0, 1);
if (bytes == 1)
{
ServerBuffer[count] = LocalBuffer[0];
count++;
if (LocalBuffer[0] == '\n')
break;
}
else
break;
string ReturnValue = oEncode.GetString(ServerBuffer, 0, count);
return ReturnValue;
}
}
NNTPException Class
Again, we define our own exception class for NNTP errors:
namespace NNTPServerException
{
public class NNTPException : System.ApplicationException
{
public NNTPException(string str) : base(str)
{
}
}
}
Sample Class Use
Using the above class is quite straightforward, as you can see from the code below (again, the Main() method for a simple console application). This takes you through the use of the different methods in the class:
static void Main(string[] args)
{
try
{
// Create NNTP object
NNTP oNNTP = new NNTP();
// Connect to a server
oNNTP.ConnectNNTP("news.testserver.com");
// Get a list of newsgroups for the server
ArrayList NewsGroupList = oNNTP.GetNewsGroupListing();
foreach (string NewsGroupEntry in NewsGroupList)
System.Console.WriteLine("Newsgroup :{0}", NewsGroupEntry);
// Now let's get the news for an article called "this article"
NewsGroupList = oNNTP.GetNews("msnews.microsoft.com");
foreach (string sArticle in NewsGroupList)
System.Console.WriteLine("{0}", sArticle);
oNNTP.PostMessage("test", "test", "test@test.com (Test User)",
"test");
oNNTP.DisconnectNNTP();
}
catch (NNTPException e)
{
System.Console.WriteLine(e.ToString());
}
catch (System.Exception)
{
System.Console.WriteLine("Unhandled Exception");
}
}
I've also included a sample NNTP application in the code download for this chapter. It works much like the POPMail sample that we saw earlier. It follows the basic structure of the POPMail sample, but instead uses our NNTP class to retrieve posting from newsgroups:
Expanding the NNTP Class
As with any bit of code, there is always room for improvement. The same improvements could be made for the NNTP class as for the POP3 class. Creating a better object representation of the NNTP message would be a place to start, as well as adding behind the scenes implementations of message retrieval and navigation. That would avoid any client applications that you may write having to explicitly work at the protocol level to work with the newsgroups.
Back in Chapter 1, we touched on the various protocols that exist for sending and receiving e-mails across the network. E-mail protocols are broad in the sense that they don't only pertain to .NET. Ever since the rise of Internet messaging, e-mail has been a cornerstone of electronic communication. I'm sure almost everyone reading this book has had some prior experience sending e-mails within a programming language, so we aren't really covering some new and exciting technology that .NET brings to the table. What is exciting is how easy and seamless .NET makes network programming, especially with regard to using the various e-mail protocols to send and retrieve e-mails, and to perform various tasks that may have been convoluted or difficult in the past. In this chapter, we are going to give a high-level overview of the various e-mail protocols and how they are accessed and used in a .NET environment. Particular examples will be given in C#, and discussion will follow.
E-mail in a Nutshell
It's quite easy to think of e-mail like ordinary mail that you may send out. The big difference is that no stamps are needed! Well, that and the fact that e-mail sending is almost instantaneous, and the postal service is usually very slow. There are various advantages to sending e-mail, though. E-mail can be sent to many recipients at once, as well as forwarded on and on very easily. However, in both cases we know that there is a starting point and an endpoint to a communication session. There is also a delivery mechanism that transfers the mail from one point to another along its mail route until it reaches its final destination. In the most generic sense, there are two components interacting in this scenario:
Message Transfer Agent (MTA)
User Agent (E-mail client)
In our real-world snail mail example, the Transfer Agent may be a postal or carrier company such as the USPS (United States Postal Service), FedEx, or UPS. The sender and receiver are quite obvious. In the electronic world, the place of the Transfer Agent is taken by the implementation of the SMTP (Simple Mail Transfer Protocol), which is responsible for directing e-mail from the starting point (sender) to the endpoint (receiver), bouncing the e-mail across various machines on the Internet. Keep in mind that this method stems from roots in TCP/IP, and is the most prevalent messaging topology used today. However, there are a number of other systems and standards that have been used for messaging in the past.
How does E-Mail Work?
When we think about how e-mails are actually sent across the Internet, we must first realize that the Internet is nothing more than hundreds of thousands of computers that are networked together, using the same standard protocols for communication. How does that fit into our e-mail situation? Endpoints are simply mailboxes or message stores on e-mail servers (although you could also view the sender and receivers as the starting and endpoints, and the message stores as holding places for the messages until they are received by a client program). A mail server is simply the machine (or virtual machine) that handles sending and receiving messages, and communicates with other machines in the process of handling e-mails.
A common scenario in a company may be to have employees using a Microsoft Outlook mail client that connects to the company's mail server (which may be running Exchange). All e-mail messages that come in and go out of that company are handled by the mail server. It acts as both the endpoint and starting point for any e-mails sent out over the Internet:
This is a simplified situation, as many companies have multiple servers to handle e-mail communications.
Notice that the mail clients (such as Outlook) communicate with the mail server using SMTP, and that the mail server also communicates with other machines over the Internet using SMTP. POP3 (Post Office Protocol) and IMAP (Internet Message Access Protocol) are also depicted in the diagram; these protocols only allow mail clients to access and retrieve e-mail. To send mail, e-mail clients use SMTP.
E-Mail Protocols
As this scenario shows, the three main topics for e-mail programming are SMTP, POP3, and IMAP. This is not to say that they are the only topics, just the ones that are most important to us developers when we are writing our applications that send and retrieve e-mail messages. This is because these are the standard protocols for sending and retrieving e-mail. We must keep in mind that these aren't the only ones, but the most commonly used and accepted methods. Other messaging standards such as X.400 exist as an alternative to TCP/IP related messaging. For more information about the X.400 protocol, see http://www.itu.int/ and http://www.alvestrand.no/x400/.
Internet mail is largely defined by a number of standards and recommendations made by companies and individuals close to Internet technology research and design. These standards are endorsed by the IMC or Internet Mail Consortium (http://www.imc.org/) and the IETF or Internet Engineering Task Force (http://www.ietf.org). The current set of Internet mail standards is composed of many related RFCs (Request For Comments), recommendations, and statements on common practices. Not all standards are fully endorsed or full IETF standards documents, but are considered stable and are used throughout the industry for developing e-mail software.
There are many RFCs available on the IMC web site (http://www.imc.org/rfcs.html). These documents are detailed and speak in much greater depth about every aspect of e-mail handling. For our purposes we are going to touch on a few of these RFCs within our discussion of various topics in this chapter. In particular, we will look at RFC 2821, which defines the SMTP protocol, and RFC 2822, which defines what an e-mail message should look like. For more information about these RFCs, take a look at the IMC web site.
SMTP
Simple Mail Transfer Protocol, or SMTP, is outlined in RFC 2821, and defines the interaction between mail servers that are transporting e-mails. It's good to note that SMTP for the most part uses TCP as a transport protocol (see RFC 1090 for SMTP over X.25). In essence, an SMTP session consists of a conversation between two machines that are trying to hand off or pass along an e-mail message. A simplified view of this is shown in the following figure:
As you can see, an SMTP session can be summed up in a simple communication between two computers in which the e-mail is relayed along. At this point, the e-mail will have bounced around the Internet to finally reach Machine 2 that is the mail handler for i-netway.com. If for instance, there was no user at this e-mail address (the address was invalid) or the mail handler didn't have any way to deliver the message, the machine would have replied that the e-mail was undeliverable.
Another View of an SMTP Session
Another way to look at an SMTP e-mail session would be to take a look at a Telnet connection to an actual e-mail server. Most ISPs disable this sort of access, as it allows for hackers and the like to easily spoof e-mails, which is not a good thing! This sample is only meant to show an example of the conversation between an SMTP e-mail server and a client. If someone at Wrox were trying to send me an e-mail via Telnet, the conversation would look something like this:
open i-netway.com 25
Trying. . . Connected to i-netway.com
220 I-NETWAY.COM - Server ESMTP (PMDF v4.3-10 #2381)
helo wrox.com
250 I-NETWAY.COM OK, WROX.COM.
mail from:
250 Address Ok.
rcpt to:
250 krowczyk@i-netway.com OK.
data
354 Enter mail, end with a single ".".
SUBJECT:E-mail Chapter
Andy, thanks for the chapter!
.
250 OK.
//by quitting, the message is sent
quit
221 Bye received. Goodbye.
See for a brief discussion of using the Microsoft Telnet client.
As you can see, the session begins with the Telnet command to open a connection to the i-netway.com e-mail server. This is followed by a few different SMTP commands that are needed to send the e-mail message, and the responses from the SMTP server. We looked at the most common SMTP commands in Chapter 5, where we built a simple SMTP client, but now let's look at the complete list.
SMTP Command Summary
Here is a quick summary of the various SMTP commands that are available. For more detailed explanations, please read RFC 2821.
Command Summary
Description
HELLO (HELO)
Identifies the SMTP client to the SMTP server.
MAIL (MAIL)
Initiates a mail transaction to deliver an e-mail to one or more mailboxes.
RECIPIENT (RCPT)
Identifies the recipient to whom the mail data is to be sent. If the data is to be sent to more than one recipient, multiple RCPT commands can be used.
DATA (DATA)
Marks the start of the mail data. The data following the DATA command is appended to the mail buffer. The mail data may contain any of the 128 ASCII character codes. The end of the data is marked by the sequence
SEND (SEND)
Initiates a mail transaction to deliver an e-mail to one or more terminals. Here it is important to note that ‘terminal' refers to a user's terminal screen. Because most clients are not terminal clients nowadays, the SEND, SOML, and SAML commands are for the most part obsolete. They are included here for the sake of completeness, because they are part of the RFC.
SEND or MAIL (SOML)
Initiates a mail transaction to deliver an e-mail to one or more terminals or mailboxes. The e-mail is delivered to the terminal of each recipient if that recipient is active on the host and accepting terminal messages, and to every recipient's mailbox.
(SAML)
Initiates a mail transaction to deliver an e-mail to one or more terminals or mailboxes. The e-mail is delivered to the terminal of each recipient if that recipient is active on the host and accepting terminal messages, and to every recipient's mailbox.
RESET (RSET)
Aborts the current mail transaction. All data will be discarded, and all buffers cleared. The receiver must send an OK reply.
VERIFY (VRFY)
Asks the receiver to confirm that the following argument is a valid username. If it is, the full name of the user (if known) and the fully specified mailbox are returned.
EXPAND (EXPN)
Asks the receiver to confirm that the following argument is a mailing list, and if so, to return the names of the members of that list. The full names of the users (if known) and the fully specified mailboxes are returned in a multi-line reply. This command is similar to the VRFY command, but is used for multiple recipients.
HELP (HELP)
Asks the receiver to send helpful information to the sender. The command can take an argument (such as a command name) and return more specific information as a response.
NOOP (NOOP)
This command has no effect, except that the receiver should send an OK reply. No operation takes place, and no other commands or data should be affected.
QUIT (QUIT)
On receiving a QUIT command, the receiver must send an OK reply, and then close the connection to the sender.
TURN (TURN)
On receiving a TURN command, the receiver must either (1) send an OK reply and take on the role of the SMTP client; or (2) send a refusal reply and retain the role of the SMTP server. This command is deprecated because of security concerns. Poor authentication implementations would allow a client machine to take on the role of the SMTP server and divert e-mail messages.
Reply Codes
If you look back at the example of sending an e-mail using SMTP through Telnet, you will see that the server sends back a reply code for each SMTP command. Actually, the reply codes sent back by the server provide us with much information about the status of the current e-mail transaction. Let's take a look at a typical reply code and see what information it contains:
Since the first two digits of the SMTP reply code are the most important to us, let's look at what they indicate. Please note that the first digit only gives the status of the command. The subsequent digits give us greater detail as to what the success or error was caused by.
First Digit
indicates preliminary acceptance of the command, pending confirmation
indicates successful completion of the command
indicates intermediate acceptance of the command, pending further information
indicates temporary negative status
indicates failure
Second Digit
syntax
connection
To see how this works, let's go over a few of the most common reply codes that we might receive back from an SMTP transaction call. The error or status messages given may differ from SMTP server to SMTP server, but the reply code means the same thing in each case. The status message is really just a human-readable description of what went wrong.
Reply Code
Message
500
Syntax error, command unrecognized
501
Syntax error in parameters or arguments
502
Command not implemented
503
Bad sequence of commands
220
221
421
250
Requested mail action OK, completed
354
Start mail input, end with
550
Requested action not taken, mailbox unavailable
553
Requested action not taken, mailbox name not allowed
554
Transaction failed
RFC 2821 defines all of the reply codes available, and the numeric codes are definitive. This means that you can't define new codes as you see fit! However, since we are talking about an agreed standard protocol, the response codes should be predefined and set in stone.
A Typical E-Mail Message
Before we start talking about accessing e-mails with POP3 and IMAP, let's first briefly touch on the content and structure of a mail message as it relates to the other important standards document that we spoke about earlier. RFC 2822 describes the content of what an e-mail message should look like. Think of it as sort of a 'schema' or outline of a typical mail message. The following is a typical mail message:
Received: from MAILCLUSTER [111.111.111.111] by mail.brinkster.com with ESMTP
(SMTPD32-6.05) id AF1F18B600FA; Fri, 28 Jun 2002 05:40:47 −0400
Received: by MAILCLUSTER with Internet Mail Service (5.5.2653.19)
id
Message-ID:
From: Wrox Press
To: "'Andrew Krowczyk'"
Subject: C# Networking Chapter
Date: Fri, 28 Jun 2002 10:39:28 +0100
Importance: high
X-Priority: 1
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain
X-RCPT-TO:
X-UIDL: 323073316
Status: U
Andy, thanks for the work you've done!
-Wrox Press
This typical e-mail structure consists of both message headers and message text. Some information is required for the message to conform to the e-mail standards while some information is optional and can be included or excluded depending on many factors.
Required Information
There are a few headers that are generally required in all e-mails:
FROM: The agent (person, system, or process) that created the message. This should be a single authenticated machine address generated by the sending agent. As you would imagine, this tells us who/what is sending us the e-mail message.
DATE: The date the message was sent. The only optional parts of the date specification are the day of the week and the seconds. The timezone may be given in usual denotations such as CST, EDT, GMT. The timezone is preferred as a numeric offset from the GMT. In the message above, you will see +0100 to denote the GMT offset.
One Recipient Address: At least one recipient address must be used, which can be either To, Cc, or Bcc.
Some Optional Information
There are also other headers and information that may be optional and not required in every message. A few of these are listed below:
REPLY-TO: The reply-to header is often used to designate the preferred e-mail address for responses to be sent to. This is often used by list mail and other processes to correctly identify the return e-mail address location.
SENDER: Why would the sender be optional? This is really the same thing as the required FROM field listed above. This field is intended for use when the sender of the e-mail is not the author of the e-mail, or is one of a group of authors. This shouldn't be used if it's identical to the FROM field. The SENDER field must be present if it's different from the FROM field.
An example of the SENDER field usage would be.
FROM: "Joe Someone"
SENDER: INFO-SAMPLE Discussion
TO: Multiple recipients of list INFO-SAMPLE Discussion
Received Headers
One of the most important parts of an e-mail message is the data that is transmitted in the received headers of the mail message. This provides debugging information, as well as a good look at where the e-mail came from and how it got from point A to point B.
Our sample above contains the following received lines:
Received: from MAILCLUSTER [111.111.111.111] by mail.brinkster.com with ESMTP
(SMTPD32-6.05) id AF1F18B600FA; Fri, 28 Jun 2002 05:40:47 −0400
Received: by MAILCLUSTER with Internet Mail Service (5.5.2653.19)
id
The received lines tell us many things:
They are the postmaster's primary debugging tool. By reading the header, we can find out much information about where the mail came from and how it got to the final destination.
They tell which systems have touched or tampered with the mail.
Each MTA (Mail Transfer Agent) that relays a message attaches its own Received header line.
RFC 2882 requires that MTAs add their own received line when they handle the mail, and they are prohibited by the RFC from touching the received lines put in by other mailers.
The received headers show us the path, hop by hop, that the mail took from the sender to the receiver.
An example of how the messages can be traced via the received lines in the headers is shown in the diagram below:
Viewing Headers in Outlook
Since the headers contain so much information, it's worth seeing quickly how to view header information from Microsoft Outlook. First, right-click on the message that you wish to view the header information for, and select Options. The Internet headers for the message are displayed at the bottom of the resulting dialog:
Note that Internet headers might not be displayed for e-mails that are sent internally on the same network.
What About MIME?
Now that we've seen the basics of an e-mail message and looked at a high-level overview of the e-mail protocol, we come to the question, 'What about attachments?' E-mail was originally a text-only messaging system using just ASCII characters. But that quickly changed into the need to 'attach' binary files of different types to the e-mail messages and have the attachments transferred along with the ASCII text messages. So, to address this need, MIME (Multi-purpose Internet Mail Extensions) was developed; MIME defines extensions to the SMTP protocol to support binary attachments of arbitrary format.
When we look at what MIME is composed of, we find that it really encompasses two main functions:
MIME encodes binary data so that it can be passed over the Internet. There are two points to notice here:
We must remember that the Internet is a 7-bit ASCII world.
8-bit extensions don't work, as there are issues with line length and file formatting.
MIME attaches a label or tag to the encoded data so that the content can be determined and interpreted at the end point of the message. For example, this is a movie file, or a Microsoft Excel document.
MIME uses a new encoding scheme that is called BASE64. It also adds new SMTP headers that describe the attached document. The idea is actually quite simple-we are just encoding the binary data into a different bit data representation and then piggybacking that data with the e-mail itself as an attached bit of data. Once the endpoint e-mail client receives the message, the new headers tell it that there is an attached document (or attached documents) embedded within the e-mail message, and the client can properly decode and display the document. The RFCs that define MIME and its composition are RFC 2045 through RFC 2049.
MIME Headers
When we take a look at the headers that must be used when MIME is used to attach a bit of information we will find that there are a few different fields that are available for use. Let's look first at the required field:
Required Fields
Description
MIME-Version
This field indicates which version of MIME is being used (currently 1.0).
Now the optional fields:
Optional Fields
Description
Content-type
This field describes what format this part of the message is in, such as text, message, application, multipart, image, audio, among others. The default type is ASCII text.
A few common content types include:
text/plain
text/html
application/binary
application/postscript
image/gif
image/jpeg
Content-transfer_encoding
This header tells how to decode the message. There are a few different encoding schemes that may be used.
Base64. This encoding is used to encode binary data in 7-bit ASCII data.
7-bit. No encoding, case insensitive.
8-bit. No encoding.
binary. No encoding.
x-token proprietary encoding designation. The x designates that this is a non-standard status encoding scheme. This simply means that the content is not encoded with a standardized coding scheme, which in most cases is bad, as it doesn't fall into interoperability guidelines.
Content-ID
ID field that allows one body to make reference to another. Similar to Message-ID (unique identifier in the e-mail message structure). Generally this field is optional.
Content-description
A textual description about the encoded data. For example, an image content type might contain 'a picture of the moon'.
Content-disposition
Field used to tell the e-mail client if the content should be displayed 'inline' with the message or as an 'attachment' to the message. There is a parameter called Filename that is a suggestion for a name if the encoded data is detached from the mail message.
Now let's take a quick look at a typical mail message that includes a MIME attachment:
From: krowczyk@i-netway.com (Andrew Krowczyk)
Subject: Sample message with Word Document Attachment
To: mailto:editor@wrox.com
MIME-version: 1.0
Content-type: MULTIPART/MIXED; BOUNDARY="Boundary_[ID_nf991kyavAuSo/HeKKQ]"-
Boundary_[ID_nf991kyavAuSo/HeKKQ]
Content-type: text/plain; charset=us-ascii
Hi, please see the attached Word Document.
-Andy-Boundary_[ID_nf991kyavAuSo/HeKKQ]
Date: Fri, 3 Jul 2002 16:43:34 −0700
Content-type: application/mac-binhex40; name=sample_worddoc.doc
Content-disposition: attachment; filename=sample_worddoc.doc
PGh0bWw+DQo8aGVhZD4NCjx0aXRsZT6q967mpcC/y7hgrKGwyjwvdG10bGU+DQo8bWV0YSBodHRw
LWVxdWl2PSJDb250ZW50LVR5cGUiIGNvbnRlbnQ9InRleHQvaHRtbDsgY2hhcnNldDliaWclIj4N
CjxsaW5rIHJlBD0ic3R5bGVzaGVldCIgaHJlZj0iaHR0cFDovL3d3dy5raW5nLmNvbS50dy9wbS5j
c3MiIHR5cGU9InRleHQvY3NzIj4NCjwvaGVhZD4NCjxzY3JpcHQgbGFuZ3VhZ2U9IkphdmFTY3Jp
cHQiPg0KZnVuY3Rpb24gZ290b3VybChkaXl0eXBlKSB7DQogdmFyIHB3PXdpbmRvdy5vcGVuKCdo
As you can see, the MIME sections and headers define the message boundaries and the binary content. For brevity, I've left out the actual binary encoding of the document. It would look like a bunch of garbled data in ASCII characters.
For the most part, I've covered only a few topics on MIME attachments. This was only to give a brief high-level overview of how attachments are included in SMTP e-mail messages. Later in the chapter, we'll look at an example of sending e-mails that include attachments from a .NET application. Hopefully, this overview of MIME will help you to understand what our .NET code is causing to happen behind the scenes when we send e-mail messages. Now we've seen what happens when we send an e-mail, we'll look at the protocols and methods for retrieving e-mail messages.
Retrieving Client-Server E-Mail
When we think about and look at how e-mail messages are retrieved from mail server message stores, we generally think of it as a client-server relationship. We use an e-mail client, such as Microsoft Outlook, Lotus Notes, or other such mail packages to retrieve messages from a central mail server mailbox. There are also web based e-mail services such as Hotmail that follow along the same lines. There are generally three models of e-mail handling:
Offline (POP3 model)
Client connects to the mail server and pulls or retrieves e-mail down to the client.
In this instance, all the mail is stored on the client machine, not the server. Usually e-mail is deleted from the server once it's retrieved, although some mail clients give the ability to 'leave messages on the server'.
Online (Original IMAP model)
The client application connects to the server for every transaction.
Everything is stored on the server.
Disconnected (Later IMAP model)
Client and server share the storage of messages.
The server is always correct, and the client application must 'synchronize' the list of messages with the server.
One of the most important points to note is that POP and IMAP only get the mail messages from the server's mailbox. As with all e-mail transactions, sending mail requires the use of SMTP. In the next few sections, we'll talk in a little more depth about the POP and IMAP protocols for retrieving e-mails. After that, we'll delve into the code required in .NET to perform all the actions we've discussed so far.
POP3
There are actually a few versions of the POP protocol that have been around. The POP2 and POP3 standards are not even compatible from a protocol standpoint. But since the huge majority of POP clients are POP3, we'll restrict our discussion to that protocol.
So what does POP3 allow us to do? As we've already learned, POP3 only allows a client to retrieve mail from a mail server. This protocol allows us to write nice-looking GUI (Graphical User Interface) applications that will present e-mail in a way that's easy to read and manage. For reference, POP3 is outlined in further detail in RFC 1939.
In the POP3 world, the mail exists on the mail server when it's received by the system. It just sits there, waiting to be picked up by a mail client. Once we connect to the mail server with a client mail program, the POP3 client copies the mail from the server to the local machine's hard drive. This method locks us into reading the mail on the client machine. Why would this make a difference? Depending on the situation, POP3 access might not be ideal. For instance, suppose we have a mail account that we read both at work and at home. Let's suppose that we have Microsoft Outlook installed on both the office and home machines. If we connect at the office and Outlook downloads four new mail messages, those messages will generally be removed from the server and transported to the local hard drive on the office machine. This means that we've lost access to those messages from our home machine. Because POP3 moves the messages from the server to your client machine, sharing a POP3 e-mail system between multiple machines can be troublesome.
POP3 Via Telnet
The only reason that I'm going to cover this is to quickly show the steps or procedures that allow us to access a typical POP3 mailbox on a mail server. The internal methods used in the .NET C# System.Web.Mail namespace would do something very similar to this while retrieving messages using the POP3 protocol. Although most of this is abstracted away by the .NET runtime components, it's interesting to see what happens behind the scenes.
open mail.someserver.com 110
Trying. . . Connected to MAIL.SOMESERVER.COM
+OK test.someserver.COM MultiNet POP3 Server Process v4.0(1) at Fri 20-Jun-2002
3:21PM-CST
user krowczyk //designate user
+OK User Name (krowczyk) ok. Password, please.
pass thisismypassword //enter password
+OK 3 messages in folder INBOX (V4.0)
list 2 //list gives message size
+OK 2 7124 //in bytes
stat //stat gives total message
+OK 3 14749 //size in bytes
quit
+OK POP# MultiNet test.somewhere.COM Server exiting (3 INBOX messages left)
Connection closed by Foreign Host
POP Commands
A typical POP session consists of a client connecting to the POP3 server on TCP port 110 (the default). Once the client is connected, the POP3 server sends back a connection-greeting message that acknowledges the connection and starts the POP3 session. Commands are then issued and responded to by the client and server until the connection has been closed and the session ends.
According to the RFC, POP3 commands consist of a keyword optionally followed by arguments. Some of the specifics for commands are as follows:
Commands are terminated by a CRLF sequence
Keywords are separated by a SPACE character
Keywords are three or four characters long
Each argument may be up to 40 characters in length
Responses may be up to 512 characters in length
+OK designates a positive response
-ERR designates a negative response or error condition
Required Commands
Description
USER [name]
Username sent to the server.
PASS [password]
The password.
QUIT
Terminates or ends the current session.
DELE [msg]
Deletes mail from the server.
RSET
Undoes any changes made during the current session.
STAT
Returns the number of messages on the server.
RETR [msg]
Retrieves the content of a message.
LIST [msg]
Returns information about the message in parameter. Such as size in bytes. If no parameter is given, a list with all messages and their sizes is returned.
NOOP
Does nothing but cause the server to respond with a positive response.
Optional Commands
Description
TOP [msg] [n]
The server sends the headers of the message, the blank line separating the headers from the body, and then the number of lines of the indicated message's body, [msg] is the message number desired, [n] specifies the top n lines to be retrieved.
UIDL [msg]
If an argument is given, the server issues a positive response with a line containing information about the specified message. Called a 'unique id listing' for the message selected. The unique-id of a message is an arbitrary server-determined string, consisting of one to 70 characters in the range 0×21 to 0×7E, which uniquely identifies a message within a maildrop.
APOP [mailbox] [digest]
A string identifying a mailbox and an MD5 digest string. The MD5 algorithm takes a message of arbitrary length as an input, and produces a 128-bit fingerprint message digest of the input. Typically used within RSA cryptography and such.
The POP3 protocol gives us basic functionality for retrieving messages from our mail server. It serves its purpose very well, and is pretty much the standard that the majority of mail servers and clients use.
IMAP
IMAP gives us everything that POP3 doesn't. It provides capability for on-line, off-line, or disconnected modes of operation. It allows us to control folders on both the client machine as well as the server. It also provides enhanced authentication (POP3 authentication is relatively weak). And it allows the use of multiple mail servers that can work together with the same mail client.
Let's take a brief look at how an IMAP client and server might interact:
IMAP4 is outlined and documented in RFC 2060. Some of IMAP's features include:
Server-side Mailbox Manipulation
Ability to add messages to a remote mailbox
Support for and notification of simultaneous updates in shared mailboxes
New mail notification
Multiple Mailbox Support
Remote folder management: list/create/delete/rename
Support for folder hierarchies
Access to message types other than e-mail, such as NetNews
Online Performance Optimization
Provision for determining message structure without downloading the entire message
Selected retrieval of MIME body parts
Server-based search and retrieval to minimize data transfer
Perfect for Roaming Power Users
Shared folders overcome limitations of POP3 access on more than one machine
Performance optimization helps with slow mail server connections
Although IMAP gives us much more advanced functionality than POP3, you'll find that many of the e-mail users and clients in the world are using POP3 servers for mail retrieval. IMAP is often much more complex to implement than POP3. For offline processing mode, POP3 and IMAP have almost equivalent possibilities, but for online or disconnected processing mode, IMAP is clearly superior. However, because of POP3's inherent simplicity, we'll slant our discussion in this chapter to cover the POP3 implementation in more detail.
.NET and E-Mails
Now that we've had an overview of mail protocols, we can get to the most interesting bits of information. Once you understand the way the SMTP, POP3, and IMAP protocols work, the code being presented here should be very easy to follow.
We'll start by looking at how to send e-mails using the .NET System.Web.Mail namespace, which allows us to send e-mail and attachments using SMTP. We'll also develop some sample applications that implement the SMTP and NNTP (the Network News Transport Protocol, which is used for newsgroup access and is very similar to SMTP).
SMTP
In your past life (when you programmed without .NET), you may have used CDONTS to send e-mails within your applications. This was commonly done from within both ASP and Visual Basic. There were often third-party ActiveX components that took the place of CDONTS and made sending e-mails a bit easier than dealing with Microsoft's confusing objects. In .NET, Microsoft has wrapped all the SMTP e-mail functionality up and rolled it directly into the .NET Framework under the System.Web.Mail namespace. Using the classes within this namespace, we can easily construct and send mail, with attachments if desired, using the SMTP service built into US.
Using the System.Web.Mail namespace actually allows us to send messages using the CDOSYS (Collaboration Data Objects for Windows 2000) message component. In fact, .NET simply wraps the functionality of the underlying messaging components. This allows the e-mail messages to be delivered either through the SMTP mail service built into Windows 2000 or through an arbitrary SMTP server.
It's been my experience in the past that third-party components often provide the deep and managed functionality that my applications have needed. In particular, if you need a component that contains more functionality than the System.Web.Mail namespace, you can take a look at the following components:
EasyMail.NET (http://www.quicksoft.com). This also provides a very nice parsing library and IMAP capabilities.
aspNetEmail (http://www.aspnetemail.com)
Smtp.NET (http://www.exclamationsoft.com/exclamationsoft/smtp.net/default.asp)
System.Web.Mail Namespace
The System.Web.Mail namespace contains three classes and three enumerations:
Class
Description
Mail Attachment
Represents the attachments of an e-mail.
MailMessage
Represents the e-mail message itself.
SmtpMail
Responsible for sending a MailMessage via SMTP.
Enumeration
Description
Mail Encoding
Specifies the encoding of the message-Base64 or UUEncode.
MailFormat
Specifies the format of the message. Either HTML or Text.
MailPriority
Specifies the priority of the message. High, Medium, or Low
Sending a message using this namespace requires us first to construct a MailMessage object to represent the e-mail. Once we've set the properties of this object and populated it with the details of the e-mail message, we send it using the SmtpMail class
You are most likely to use these various properties when initializing the MailMessage object:
MailMessage Class Properties
Description
Attachments
The list of attachments (MailAttachment objects) that are transmitted with the e-mail.
Bcc
A semicolon-delimited list of e-mail addresses that receive a Blind Carbon Copy (Bcc) copy of the e-mail.
Body
The body of the e-mail.
BodyFormat
Specifies the MailFormat of the e-mail. This can be either MailFormat.Text or MailFormat.Html.
Cc
A semicolon-delimited list of e-mail addresses that receive a Carbon Copy (Cc) of the e-mail.
From
The e-mail address of the sender.
Priority
Specifies the MailPriority of the e-mail.
Subject
The subject of the e-mail.
To
The e-mail address of the recipient.
UrlContentBase
Gets or sets the Content-Base HTTP header. The URL base of all relative URLs used in an HTML body.
UrlContentLocation
Gets or sets the Content-Location HTTP header for the e-mail message.
Headers
Read-only property that consists of the list of custom headers to be transmitted with the message.
Constructing a MailMessage Object
The first step in constructing a MailMessage object is to set a reference to the System.Web.dll assembly, since the System.Web and System.Web.Mail namespaces aren't available until the reference is set:
After we have added the reference to System.Web.dll, we can include it in a using statement, and instantiate our MailMessage object:
using System.Web.Mail;
// Create a mail message object
MailMessage email = new MailMessage();
Next, we'll set up a few properties of the e-mail:
// Set message parameters
email.From = "someone@someone.com";
emai1.To = "krowezyk@i-netway.com";
email.Subject = "Test message using SmtpMail";
email.BodyFormat = MailFormat.Text;
email.Body = "This is just a test message";
Adding an Attachment
Adding an attachment is as easy as creating one or more MailAttachment objects and adding them to the Attachments list of the MailMessage object. Here's a quick look at the members of the MailAttachment class:
MailAttachment Class Properties
Description
Encoding
Type of encoding for the e-mail attachment. This can be either MailEncoding.Base64 or MailEncoding.UUEncode.
Filename
Name of the attachment file.
Now let's add an attachment. We do this by calling the Add() method of the Attachments collection, passing in the path of the file that we want to attach to the mail, and the MailEncoding to use:
email.Attachments.Add(new MailAttachment(@"c:\testfile.txt",
MailEncoding.Base64));
We can add any number of attachments to the e-mail message in this way by continuing to add Mail Attachment objects to the Attachments list of the MailMessage.
An SmtpMail Object
To send an e-mail message that you've composed, you need to call a method of the SmtpMail object. The interesting thing to note is that there is only one really important method of the SmtpMail class- the static Send() method. This method sends off the mail message to the SMTP server. It can take a MailMessage object as a parameter, or it can take four strings that specify the sender, recipient, subject, and body of the e-mail.
The SmtpMail class also has one public property:
SmtpMail Class Properties
Description
SmtpServer
This is the server to which the SMTP mail should be sent. If this property is not set, the SmtpMail object defaults to sending the mail to the Windows SMTP service for delivery.
Please note that you must have the SMTP service installed on your machine before this code will work when not specifying an SMTP server. To install the US SMTP service, select Control Panel | Add Remove Programs | Add Remove Windows Components | Internet Information Services | SMTP Service.
// Send the e-mail using the SmtpMail object
SmtpMail.Send(email);
//or also like this - without a mail message object
string from = "from@somewhere.com";
string to = "to@somewhere.com";
string subject = "test";
string body = "test message";
SmtpMail.Send (from, to, subject, body);
If we don't set the SmtpServer property, the e-mail will be sent via the Windows SMTP service. To set the SmtpServer property we can just use the following code:
// For example, here we want to set the SMTP server field before sending the mail
// message
SmtpMail.SmtpServer = "mail.testserver.com";
// Now send message as we did before
SmtpMail.Send(email);
The previous code snippets have shown the various parts of sending e-mail using the SMTP capabilities of .NET. Now we'll look at a sample application that ties it all together.
SMTP Mailer Application
Our finished sample SMTP mail application will be a simple .NET Windows Form application that allows us to set various properties of an e-mail and send it using a designated SMTP server. The following screenshot depicts the end result of the code we're going to cover. The form has five single-line text boxes, where the user can enter the name of the SMTP server to use, the e-mail addresses of the sender and recipient, the subject of the mail, and the path to any attachment to send with the mail. There is also a multi-line text box, where the body of the mail can be typed:
This code is available in the chapter download materials. There is really no magic to writing this sort of simple application. We basically just create a form with appropriate text boxes and enter some code in the Click event handler for the Send button that uses the SMTP mail capabilities we've been looking at to send the mail on its way.
Important Code
As before, we need to add a reference to the System.Web.dll assembly, and a using directive to point to the System.Web.Mail namespace. Apart from the standard Windows code (generated by Visual Studio .NET), we only need to add the code behind the Send button. This is very similar to the code we've already seen:
private void button1_Click(object sender, System.EventArgs e)
{
// Create mail message
MailMessage email = new MailMessage();
// Set message parameters
email.From = txtFrom.Text;
email.To = txtRecipient.Text;
email.Subject = txt Subject.Text;
email.BodyFormat = System.Web.Mail.MailFormat.Text;
email.Body = txtMessage.Text;
// Add attachment
email.Attachments.Add(new MailAttachment(txtAttachment.Text,
MailEncoding.Base64))
// Set SMTP server
SmtpMail.SmtpServer = txtSmtpServer.Text;
// Now send message
SmtpMail.Send(email);
}
Check out the download materials for the full sample application.
POP3
If you've had any experience with .NET mail programming or development, you will probably know that .NET doesn't provide inherent support for retrieving mail from POP3 and IMAP mailboxes. Although we've already covered the SMTP component of the .NET Framework that merrily allows us to send SMTP e-mails and so on, what we haven't said yet is that the System.Web.Mail namespace is basically just a wrapper for the CDOSYS component. This unmanaged component is really used underneath to send the e-mails.
So how do we retrieve mails using .NET? We could try using CDO to retrieve mails, but that will only work for Microsoft Exchange Servers, and not all the mail servers out there in the world are based on Microsoft Exchange! What we really want to do, seeing that this is a networking book, is to code our own POP3 e-mail class that uses the POP3 protocol to retrieve e-mails. We'll use some of the networking namespaces that exist in .NET to accomplish this goal.
Creating a POP3 C# Class
Our first task in creating a class that handles POP3 messages is to determine what exactly a message consists of. First create a new C# Windows Application project called POPMail.
Let's start off by creating a class that represents the core of a POP3 e-mail message, so add a new class to the project called POP3EmailMessage:
public class POP3EmailMessage
{
// Define public members
public long msgNumber;
public long msgSize;
public bool msgReceived;
public string msgContent;
}
This class contains a message number, the size of the message (in bytes), a flag indicating whether or not the message has been received from the server, and a string containing the actual content of the e-mail message. If you remember in our protocol discussion, these fields are common within a typical e-mail message and are used/shown when retrieving a message. For example, we need an index (msgNumber) to the message that we want to deal with when listing out or retrieving from the POP3 server.
System.Net.Sockets.TcpClient
As we've seen in the last few chapters, the .NET Framework presents us with a rich base of classes to deal with low-level networking needs. In our case, we want to harness the TcpClient class, because we will make remote connections to the mail server using the TCP protocol. We will therefore create a class that is derived from the System.Net.Sockets.TcpClient namespace:
// Define the POP3 class
public class POP3 : System.Net.Sockets.TcpClient
{
Connecting to a Server
There are a few things that this class must be able to do, the first of which is connect to the POP3 server passing in a username and password.
Remember our Telnet session back in our discussion about the POP3 protocol to draw similarities to what we are doing here.
So, let's start by writing a ConnectPOP() method. This takes three parameters-the name of the server to connect to, and the username and password of the mailbox we want to access. Note that it will call some methods that are presented later in the chapter:
public void ConnectPOP(string sServerName, string sUserName, string sPassword)
{
// Message and the server resulting response
string sMessage;
string sResult;
// Call the connect method of the TcpClient class
Connect(sServerName, 110);
// Remember default port for server is 110
// Get result back
sResult = Response();
// Check response to make sure it's +OK
if (sResult.Substring(0,3) != "+OK")
throw new POPException(sResult);
// Got past connect, send username
sMessage = "USER " + sUserName + "\r\n";
// Write() sends data to the Tcp Connection
Write(sMessage);
sResult = Response();
// Check response
if (sResult.Substring(0,3) != "+QK")
{
throw new POPException(sResult);
}
// Now follow up with sending password in same manner
sMessage = "PASS " + sPassword + "\r\n";
Write(sMessage);
sResult = Response();
if (sResult.Substring(0,3) != "+OK")
throw new PQPException(sResult);
}
I've put comments in the above code to help with its readability. We connect to the POP3 server and send the username and password using the Connect() method of the TcpClient class, and our methods Write() and Response(). As we saw in the POP3 Telnet session that we discussed earlier, the POP3 server should send back a +OK response if we were successful, and a -ERR message when a failure occurs. If this happens, we throw an exception using the message sent back from the server.
When You Connect, You Must Disconnect
Since we're talking about connecting, we may as well also show the disconnect method now. To disconnect from the server, we simply need to issue the QUIT command. That makes our DisconnectPOP() method really quite simple:
public void DisconnectPOP()
{
string sMessage;
string sResult;
sMessage = "QUIT\r\n";
Write(sMessage);
sResult = Response();
if (sResult.Substring(0,3) != "+OK")
throw new POPException(sResult);
}
So far, we have a way to connect to the POP3 server, as well as a way to disconnect from it. The typical POP3 session will consist of a call to ConnectPOP(), commands to get e-mails and so on, and then a call to DisconnectPOP().
Getting a List of Messages
We can get a list of messages on the POP3 inbox simply by issuing a LIST command to the server. The ListMessages() method shows how we can do this.
Important Note that using the ArrayList requires us to import the System.Collections namespace.
public ArrayList ListMessages()
{
// Same sort of thing as in ConnectPOP and DisconnectPOP
string sMessage;
string sResult;
ArrayList returnValue = new ArrayList();
sMessage = "LIST\r\n";
Write(sMessage);
sResult = Response();
if (sResult.Substring(0, 3) != "+OK")
throw new POPException (sResult);
while (true)
{
sResult = Response();
if (sResult == ".\r\n")
{
return returnValue;
}
else
{
POP3EmailMessage oMailMessage = new POP3EmailMessage()
// Define a separator
char[] sep = { ' '};
// Use the split method to break out array of data
string[] values = sResult.Split(sep);
// Put data into oMailMessage object
oMailMessage.msgNumber = Int32.Parse(values[0]);
oMailMessage.msgSize = Int32.Parse(values[1]);
oMailMessage.msgReceived = false;
returnValue.Add(oMailMessage);
continue;
}
}
}
As we saw in the Telnet example, sending a LIST command to the POP3 server will cause the server to send back multiple lines of text. Each line represents an e-mail message that contains a message number and a number of bytes that the e-mail message contains. It's key to note that this method only returns an array of message objects that have very minimal data in them. The way that we receive more information and the message content is by creating a mirror of the RETR command that retrieves the actual message.
Retrieving a Specific Message
To retrieve a full message from the POP3 server, we need to issue the RETR command with the msgNumber of the message that we want to retrieve. This follows the same procedure as we described before:
public POP3EmailMessage RetrieveMessage(POP3EmailMessage msgRETR)
{
string sMessage;
string sResult;
// Create new instance of object and set new values
POP3EmailMessage oMailMessage = new POP3EmailMessage ();
oMailMessage.msgSize = msgRETR.msgSize;
oMailMessage.msgNumber = msgRETR.msgNumber;
// Call the RETR command to get the appropriate message
sMessage = "RETR " + msgRETR.msgNumber + "\r\n";
Write(sMessage);
sResult = Response();
if (sResult.Substring(0, 3) != "+OK")
throw new POPException (sResult);
// Set the received flag equal to true since we got the message
oMailMessage.msgReceived = true;
// Now loop to get the message text until we hit the "." end point
while (true)
{
sResult = Response();
if (sResult == ".\r\n")
break;
else
oMailMessage.msgContent = sResult;
}
return oMailMessage;
}
Deleting a Message
Since the POP3 protocol states that a message isn't deleted from the server when the message is retrieved, we need to call the DELE command explicitly to remove the message from the server. Writing that method is also quite simple:
public void DeleteMessage(POP3EmailMessage msgDELE)
{
string sMessage;
string sResult;
sMessage = "DELE " + msgDELE.msgNumber + "\r\n";
Write(sMessage);
sResult = Response();
if (sResult.Substring(0, 3) != "+OK")
throw new POPException(sResult);
}
The Write() Method
The write method takes a message as input and writes it out to the TCP network stream. This will in effect send our command to the POP3 server we are connected to. Since C# string data types cannot be directly buffered to the network stream, we must use the ASCIIEncoding class of the System.Text namespace to get the byte representation of the string data. Once this is done, we just write that out to the network stream:
private void Write(string sMessage)
{
// Used for Data Encoding
System.Text.ASCIIEncoding oEncodedData = new System.Text.ASCIIEncoding() ;
// Now grab the message into a buffer for sending to the TCP network stream
byte[] WriteBuffer = new byte[1024];
WriteBuffer = oEncodedData.GetBytes(sMessage) ;
// Take the buffer and output it to the TCP stream
NetworkStream NetStream = GetStream();
NetStream.Write(WriteBuffer, 0, WriteBuffer.Length);
}
The Response() Method
The flip side of the Write() method described above is the Response() method. This method allows us to read data back from the POP3 server connection. This is used throughout the class to grab the result codes sent back from the server in response to commands that we've sent with each Write(). Again, we need to use the ASCIIEncoding class to get the string representation of the bytes being received across the network stream.
This code sample should be quite self-explanatory:
private string Response()
{
System.Text.ASCIIEncoding oEncodedData = new System.Text.ASCIIEncoding();
byte []ServerBuffer = new Byte[1024];
NetworkStream NetStream = GetStream();
int count = 0;
// Here we read from the server network stream and place data into
// the buffer (to later decode and return)
while (true)
{
byte []buff = new Byte[2];
int bytes = NetStream.Read( buff, 0, 1 );
if (bytes == 1)
{
ServerBuffer [count] = buff[0];
count++;
if (buff[0] == '\n')
{
break;
}
}
else
{
break;
}
}
// Return the decoded ASCII string value
string ReturnValue = oEncodedData.GetString(ServerBuffer, 0, count )
return ReturnValue;
}
PopException Class
This class simply encapsulates an application exception that we may throw in our code:
namespace POPMailException
{
public class POPException : System.ApplicationException
{
public POPException(string str) : base(str)
{
}
}
}
A Quick Console Application Sample
The full sample application given here can be found in this book's download from www.wrox.com. The sample application adds a GUI front end to the POP3 class that we just wrote, but the basic use of this class can be seen by taking a look at the Main() method of the console application listed below. This method steps through the items needed to instantiate the class and retrieve the message numbers and message text:
static void Main(string[] args)
{
try
{
POP3 oPOP = new POP3();
oPOP.ConnectPOP("mail.someserver.com", "username", "password");
ArrayList MessageList = oPOP.ListMessages();
foreach (POP3EmailMessage POPMsg in MessageList)
{
POP3EmailMessage POPMsgContent = oPOP.RetrieveMessage(POPMsg);
System.Console.WriteLine("Message {0}: {1}",
POPMsgContent.msgNumber, POPMsgContent.msgContent);
}
oPOP.DisconnectPOP();
}
catch (POPException e)
}
System.Console.WriteLine(e.ToString());
}
catch ( System.Exception e)
{
System.Console.WriteLine(e.ToString());
}
}
}
The image below is a screenshot of the sample PopMail application that I've written. It uses the POP3 class above to list the e-mails in my mailbox. The user can then retrieve the message text by inserting the number of the message to display. The entire message is then retrieved from the mailbox:
Expanding on this Class
There are many ways in which this class could be improved and expanded upon. You may want to expand the POP3EmailMessage class to include much more functionality. Individual properties that describe the headers, subject, message, and so on. I made the class simple to use and tried to follow the same sort of design as you would see via logging into a Telnet session with the POP3 server, as this stays more true to the design of the POP3 protocol and is useful for our understanding in a networking frame of mind.
There are also many popular third-party components that encapsulate this functionality very nicely. But what fun would buying one of those be?
NNTP
NNTP is commonly used to access the content of newsgroups on the Internet. The protocol is defined in RFC 977, and has been around for a while. We'll briefly cover a class that encapsulates access to NNTP newsgroups and servers, which is based upon and quite similar to the POP3 class that we described earlier. Why is it similar? Firstly, it's a basic TCP network stream-based protocol, much like SMTP and other e-mail related protocols. We haven't really touched much on NNTP up until this point in a protocol discussion, since this chapter is really targeted towards the topic of .NET and e-mail access. Therefore, if you'd like to read further information about the NNTP protocol, I'd advise reading RFC 977 located at http://www.ietf.org/rfc/rfc0977.txt?number=977 for further information. For ease of reading the code, I will list the common commands and responses that may be used against an NNTP server. These are very similar to the POP3 items listed in our previous examples.
NNTP Commands
The most common commands issued to an NNTP newsgroup server are listed in the following table:
NNTP Commands
Description
ARTICLE
Displays the header, a blank line, then the body text of the current or specified article.
GROUP
Will return the article numbers of the first and last articles in the group, and an estimate of the number of articles on file in the group.
LAST
The internally maintained 'current article pointer' is set to the previous article in the current newsgroup.
LIST
Returns a list of valid newsgroups and associated information.
NEWSGROUPS
A list of newsgroups created since a specified date and time will be listed in a similar format to the LIST command.
NEWNEWS
A list of message IDs of articles posted or received in the specified newsgroup since the date specified.
NEXT
The internally maintained 'current article pointer' is advanced to the next article in the current newsgroup. If no more articles remain in the current group, an error message is returned and the article remains selected.
POST
If posting is allowed, the article is posted to the server.
QUIT
Closes the connection with the server.
NNTP Responses
The most common responses received back from an NNTP newsgroup server are listed below. These are not specific messages, but a listing of the meaning of different digits that compose the NNTP response. For more complete response listings, please view the RFC.
NNTP Responses
Description
1xx
Informative message
2xx
Command OK
3xx
Command OK so far, send the rest of it
4xx
Command was correct, but couldn't be performed for some reason
5xx
Command unimplemented, or incorrect, or a serious program error occurred
x0x
Connection, setup, and miscellaneous messages
x1x
Newsgroup selection
x2x
Article selection
x3x
Distribution functions
x4x
Posting
x8x
Nonstandard (private implementation) extensions
x9x
Debugging output
Some common response codes include:
Common Messages
Description
100
Help text
190 through 199
Debug output
200
Server ready-posting allowed
201
Server ready-no posting allowed
400
Service discontinued
500
Command not recognized
501
Command syntax error
502
Access restriction or permission denied
503
Program fault-command not performed
You'll see more specific responses and commands as we discuss the NNTP code class below.
Creating an NNTP Class In C#
This explanation of the class will follow the same sort of presentation as the POP3 class. We will derive the NNTP class from the same System.Net.Sockets.TcpClient class. First we import the necessary namespaces:
using System;
using System.Net.Sockets;
using NNTPServerException; // Our own exception class implementation
using System.Collections;
Inheriting from the TCP Client Class
As in the POP3 client, inheriting from the TcpClient gives us a wealth of functionality that we don't need to implement ourselves, such as the network transfer layers responsible for connecting to the server and allowing us to send data across the network stream.
public class NNTP : System.Net.Sockets.TcpClient
Connecting to the Server
Our ConnectNNTP() method simply calls the TcpClient's Connect() method, passing in the server name as well as the standard port number (119) for the NNTP server connection:
public void ConnectNNTP(string sServer)
{
string sResult;
// Connect to the server on the default port #119
Connect(sServer, 119);
sResult = Response();
// In this case, a response code of 200 is an OK response
if (sResult.Substring(0, 3) != "200")
throw new NNTPException(sResult);
}
Disconnecting from the Server
Again, if we write a connect function, we must write a disconnect function. This simply sends the QUIT command to the NNTP server:
public void DisconnectNNTP()
{
string sMessage;
string sResult;
// Send the QUIT command
sMessage = "QUIT\r\n";
Write(sMessage);
sResult = Response();
// We expect a code of 205 acknowledging the quit
if (sResult.Substring(0, 3) != "205")
throw new NNTPException (sResult);
}
Getting the Newsgroups
As you might imagine, calling GetNewsGroupListing() in our class will use the LIST command to return all of the groups that are available on the NNTP server. This may be a lot, depending on the server!
public ArrayList GetNewsGroupListing()
{
string sMessage;
string sResult;
// Create an array for the return values
ArrayList ReturnValue = new ArrayList();
sMessage = "LIST\r\n";
Write(sMessage);
// Check the response, if OK continue
sResult = Response();
if (sResult.Substring(0, 3) 1= "215")
throw new NNTPException(sResult);
// While there are more results, loop and append to output array list
while (true)
{
sResult = Response();
if (sResult == ".\r\n" || sResult == ".\n")
{
return ReturnValue;
}
else
{
char[] separator = { ' '};
string[] values = sResult.Split(separator);
ReturnValue.Add(values[0]);
continue;
}
}
}
Getting News from a Group
To get the news from a specific newsgroup that is listed on the NNTP server, we simply need to create a method that calls the GROUP command, passing in the name of the newsgroup for which we want to retrieve the messages:
public ArrayList GetNews(string sNewsGroup)
{
string sMessage;
string sResult;
ArrayList ReturnValue = new ArrayList();
sMessage = "GROUP " + sNewsGroup + "\r\n";
// Write the message to the server
Write(sMessage);
sResult = Response();
// Check for successful operation
if (sResult.Substring(0, 3) != "211")
throw new NNTPException(sResult);
char[] separator = { ' ' };
string[] values = sResult.Split(separator);
// For beginning and end
long begin = Int32.Parse(values[2]);
long end = Int32.Parse(values[3]);
if (begin + 100 < end && end > 100)
begin = end - 100;
for (long i = begin; i
sMessage = "ARTICLE " + i + "\r\n";
Write(sMessage);
sResult = Response();
if (sResult.Substring( 0, 3) == "423")
continue;
if (sResult.Substring( 0, 3) != "220")
throw new NNTPException(sResult);
string sArticle = "";
while (true)
{
sResult = Response();
if (sResult == ".\r\n")
break;
if (sResult == ".\n")
break;
if (sArticle. Length < 1024)
sArticle += sResult;
}
ReturnValue.Add(sArticle);
}
return ReturnValue;
}
Posting to a Group
Posting to a newsgroup is also an easy task. It consists of simply calling the POST command with the name of the newsgroup, followed by the headers and the body of the message we want to post:
public void PostMessage(string sNewsGroup, string sSubject, string sFrom,
string sContent)
{
string sMessage;
string sResult;
sMessage = "POST " + sNewsGroup + "\r\n";
Write(sMessage);
sResult = Response();
if (sResult.Substring( 0, 3) != "340")
throw new NNTPException(sResult);
// Build message
sMessage = "From:" + sFrom + "\r\n"
+ "Newsgroups: " + sNewsGroup + "\r\n"
+ "Subject: " + sSubject + "\r\n\r\n"
+ sContent + "\r\n.\r\n";
Write(sMessage);
sResult = Response();
if (sResult.Substring( 0, 3) != "240")
throw new NNTPException(sResult);
}
The Write() Method
Again, due to the encoding/decoding that must take place to allow the C# string type to be transported in bytes over the network stream, we need to come up with our own method that writes the data in a memory buffer to the server:
private void Write(string sMessage)
{
System.Text.ASCII Encoding oEncode = new System.Text.ASCIIEncoding() ;
byte[] WriteBuffer = new byte[1024];
WriteBuffer = oEncode.GetBytes(sMessage);
NetworkStream oNetworkStream = GetStream();
oNetworkStream.Write(WriteBuffer, 0, WriteBuffer.Length);
}
The Response() Method
We also need to transform the data coming back from the server to an appropriate string format for our class representation:
private string Response ()
{
System.Text.ASCIIEncoding oEncode = new System.Text.ASCIIEncoding();
byte []ServerBuffer = new byte[1024];
NetworkStream oNetworkStream = GetStream();
int count=0;
while (true)
{
byte []LocalBuffer = new Byte[2];
int bytes = oNetworkStream.Read(LocalBuffer, 0, 1);
if (bytes == 1)
{
ServerBuffer[count] = LocalBuffer[0];
count++;
if (LocalBuffer[0] == '\n')
break;
}
else
break;
string ReturnValue = oEncode.GetString(ServerBuffer, 0, count);
return ReturnValue;
}
}
NNTPException Class
Again, we define our own exception class for NNTP errors:
namespace NNTPServerException
{
public class NNTPException : System.ApplicationException
{
public NNTPException(string str) : base(str)
{
}
}
}
Sample Class Use
Using the above class is quite straightforward, as you can see from the code below (again, the Main() method for a simple console application). This takes you through the use of the different methods in the class:
static void Main(string[] args)
{
try
{
// Create NNTP object
NNTP oNNTP = new NNTP();
// Connect to a server
oNNTP.ConnectNNTP("news.testserver.com");
// Get a list of newsgroups for the server
ArrayList NewsGroupList = oNNTP.GetNewsGroupListing();
foreach (string NewsGroupEntry in NewsGroupList)
System.Console.WriteLine("Newsgroup :{0}", NewsGroupEntry);
// Now let's get the news for an article called "this article"
NewsGroupList = oNNTP.GetNews("msnews.microsoft.com");
foreach (string sArticle in NewsGroupList)
System.Console.WriteLine("{0}", sArticle);
oNNTP.PostMessage("test", "test", "test@test.com (Test User)",
"test");
oNNTP.DisconnectNNTP();
}
catch (NNTPException e)
{
System.Console.WriteLine(e.ToString());
}
catch (System.Exception)
{
System.Console.WriteLine("Unhandled Exception");
}
}
I've also included a sample NNTP application in the code download for this chapter. It works much like the POPMail sample that we saw earlier. It follows the basic structure of the POPMail sample, but instead uses our NNTP class to retrieve posting from newsgroups:
Expanding the NNTP Class
As with any bit of code, there is always room for improvement. The same improvements could be made for the NNTP class as for the POP3 class. Creating a better object representation of the NNTP message would be a place to start, as well as adding behind the scenes implementations of message retrieval and navigation. That would avoid any client applications that you may write having to explicitly work at the protocol level to work with the newsgroups.
Subscribe to:
Posts (Atom)
