Agilent Developer Network - White Paper

Agilent Developer Network - White Paper

Agilent Developer Network White Paper Storing Test Data to File Using Microsoft® Visual Basic® .NET December 2002 Keith Hill Agilent Technologies Summary: This white paper introduces the .NET Framework Class Library’s file I/O functionality that is available to Microsoft® Visual Basic .NET programmers. Five examples are presented that demonstrate various ways to save and retrieve test data from a file. In addition, examples are presented to demonstrate reading application configuration files, file and directory manipulation and monitoring files. Contents Introduction Saving Test Data Using Comma Separated Value Text Format Saving Test Data Using Name Value Pair Text Format Saving Data Using BinaryWriter Saving Data Using Binary Serialization Saving Data Using XML Serialization Reading Application Configuration Settings File and Directory Operations Monitoring Files and Directories with FileSystemWatcher Conclusions Introduction Microsoft Visual Basic .NET (VB.NET) is a significant new release and a worthy successor to Visual Basic 6.0.

VB.NET introduces language advances, a new runtime engine and a significantly more powerful runtime library. The language changes introduce full support for object-oriented programming and structured exception handling. The new runtime engine, called the Common Language Runtime (CLR) provides better memory management and support for multiple threads. The .NET Framework Class Library (FCL) provides significantly more functionality than the VB 6 runtime library. In particular, the FCL provides more options for storing test data to file and for manipulating files and directories in general. Before exploring the new file I/O functionality in the FCL, it is worth pointing out that the VB 6 approach to file I/O using VB 6 statements like Open, Print, Input, Close is still available in VB.NET albeit with some syntax changes.

For example, in VB 6 you would open a file, read from it and close it with the code shown in Figure 1.

Agilent Developer Network White Paper 2 of 17 Storing Test Data to File Using Microsoft® Visual Basic® .NET November 2002 Figure 1 Reading a File using VB 6 Dim reading As Double Open "C:\Temp\TestData.txt" For Input As #1 Input #1, reading Close #1 In VB.NET, this code changes slightly as shown in Figure 2. Figure 2 Reading a File using VB 6 Compatibility Functions in VB.NET Dim reading As Double FileOpen(1, "C:\Temp\TestData.txt", OpenMode.Input) Input(1, reading) FileClose(1) The VB 6 file I/O statements have been replaced by a set of compatibility functions including FileOpen, Print, Input and FileClose among others.

For more information see File Access Types, Functions, and Statements. Despite the syntax changes, performing file I/O with these functions is basically the same as it was in VB 6. You can continue to use that approach in VB.NET if you desire. Keep in mind that the VB 6 compatibility methods mentioned above are merely wrappers on top of the FCL’s native file I/O functionality. Besides just bypassing an extra layer of code, learning how to directly use the FCL’s file I/O objects and methods gives you more flexibility, an easier mechanism for saving test data and allows you to read file I/O code in other .NET languages like C#.

The rest of this article explores those native FCL file I/O objects and methods.

The 1.0 version of the FCL contains more than 3000 public types that contribute over 16000 public properties, 33000 public methods and 4000 public events (I told you it contained significantly more functionality than the VB 6 runtime library). Fortunately this functionality is organized into many different namespaces like System, System.Collection, System.Net, System.Text, System.Threading, System.Windows.Forms just to name a few. The file I/O functionality is located in the System.IO namespace. There are a number of file I/O examples used throughout the rest of this article. All of the examples dealing with test data use the data format shown in Figure 3 to illustrate the different ways to save and retrieve test data.

Figure 3 Test Data Elements Name Data Type Description dutId String Device under test identifier timestamp DateTime Time data was acquired stationId Integer Test station identifier data Double() 3 test data values The example code associated with this paper can be downloaded from http://www.agilent.com/find/adnwhitepaperexamples. Visit the Agilent Developer Network at http://www.agilent.com/find/ADN

Agilent Developer Network White Paper 3 of 17 Storing Test Data to File Using Microsoft® Visual Basic® .NET November 2002 Saving Test Data Using Comma Separated Value Text Format One very popular text file format is “comma separated value” or CSV. CSV files can be easily imported by applications like Microsoft Excel. Figure 4 shows how to save data in a text file using CSV format. Figure 4 Writing Data to a CSV Text File ' Example 1a in the sample project ' Create test data Dim dutId As String = "A00100" Dim timestamp As DateTime = DateTime.Now Dim stationId As Integer = 1 Dim data() As Double = New Double() {1.1, 2.2, 3.3} ' Open file for writing text Dim writer As StreamWriter = File.CreateText(filename) ' Write data in csv (comma separated value) format: ' DutIdentifier,Timestamp,StationID,DataUpperBound,Data() writer.Write("{0},{1},{2},{3}", dutId, timestamp, _ stationId, UBound(data)) Dim val As Double For Each val In data writer.Write & val) Next ' Close StreamWriter when done writer.Close() The following line of code from Figure 4 demonstrates the use of the System.IO.File class to open up a file for writing in text mode.

Dim writer As StreamWriter = File.CreateText(filename) One interesting tidbit about the File.CreateText method is that it uses something called UTF-8 encoding for the text written to file. The advantage to UTF-8 encoding is that ASCII characters require only one byte per character in the file. At the same time, Unicode characters can be saved although these characters may occupy two or more bytes per character in the file. There are ways to force exclusive ASCII or Unicode encoded text files. You will see an example of how to do this later. StreamWriter is another class in the System.IO namespace.

It defines a number of overloaded methods for writing various primitive data types in text format. Notice that the data array is written out preceded by the upper bound. Writing the upper bound is not really necessary especially if the array is a fixed size. However, specifying the upper bound does make it easier to read the data from file and it allows you to change the size of your arrays without breaking code that reads your file format. When the code in Figure 4 executes it produces the file shown in Figure 5.

Visit the Agilent Developer Network at http://www.agilent.com/find/ADN

Agilent Developer Network White Paper 4 of 17 Storing Test Data to File Using Microsoft® Visual Basic® .NET November 2002 Figure 5 CSV File Contents A00100,10/11/2002 2:20:35 AM,1,2,1.1,2.2,3.3 The code to read the test data from this file is shown in Figure 6. Figure 6 Reading from a CSV Text File ' Example 1b in the sample project ' Open file for reading as text Dim reader As StreamReader = File.OpenText(filename) ' Read entire file as a string and split the string on commas Dim strings As String() = reader.ReadToEnd().Split(",") ' Parse each individual string Dim dutId As String = strings(0) Dim timestamp As DateTime = DateTime.Parse(strings(1)) Dim stationId As Integer = Integer.Parse(strings(2)) Dim data() As Double = New Double(Integer.Parse(strings(3 { } Dim ndx As Integer For ndx = 4 To strings.Length - 1 data(ndx - 4) = Double.Parse(strings(ndx)) Next ' Close StreamReader when done reader.Close() The following line of code from Figure 6 opens a file for reading in text mode.

Dim reader As StreamReader = File.OpenText(filename) Then the code reads in the entire file using the StreamReader.ReadToEnd method. This approach is fine for smaller files but it can use large amounts of memory for large data files. I will show you a better approach to reading a large text file in the next example. Since the file is in CSV format, this example uses the String.Split method to split the string on commas. The implicit assumption here is that none of our string values contain commas. That leaves an array of strings to parse. Since the dutId variable is of type string no parsing is required.

For timestamp, the DateTime class fortunately provides a shared Parse method that takes a string containing a date and time and creates a new DateTime object. As it turns out, most of the VB.NET primitive types like Integer and Double also provide shared Parse methods that take a string and return a value of the appropriate type. As a result, reading the stationId is straightforward. For the array, this example parses the array size first and allocates the appropriate size array. The following line of code may look a bit unusual to a VB 6 programmer.

Dim data() As Double = New Double(Integer.Parse(strings(3 { } Visit the Agilent Developer Network at http://www.agilent.com/find/ADN

Agilent Developer Network White Paper 5 of 17 Storing Test Data to File Using Microsoft® Visual Basic® .NET November 2002 This line of code allocates an array based on the array size that is read from the file via strings(3) without requiring a ReDim. Then the example iterates through the rest of the strings converting each string into a double value to put into the data array. Saving Test Data Using Name Value Pair Text Format One of the problems with CSV format is that if you want to store strings that contain embedded commas, then parsing the file contents becomes much harder.

Certainly String.Split wouldn’t work very well in this case. Another approach to saving data that doesn’t suffer from problems handling special characters like commas is “name value” format. Furthermore, since each test data element is saved on its own line in the file, it is easier to read the file in smaller chunks versus reading the entire file at once. Figure 7 demonstrates how to save the same test data in name value pair format.

Figure 7 Writing Data to a Name Value Pair Text File ' Example 2a in the sample project ' Create test data Dim dutId As String = "A00100" Dim timestamp As DateTime = DateTime.Now Dim stationId As Integer = 1 Dim data() As Double = New Double() {1.1, 2.2, 3.3} ' Open file for writing text with ASCII encoding Dim writer As StreamWriter = _ New StreamWriter(filename, False, Encoding.ASCII) ' Write data as name=value pairs, one per line writer.WriteLine("DutId=" & dutId) writer.WriteLine("Timestamp=" & timestamp) writer.WriteLine("StationId=" & stationId) writer.Write("Data=") Dim ndx As Integer For ndx = 0 To UBound(data) writer.Write(data(ndx)) If ndx UBound(data) Then writer.Write(",") End If Next ' Close StreamWriter when done writer.Close() Notice in this example that a StreamWriter object is created directly rather than using File.CreateText as shown in this line of code.

Dim writer As StreamWriter = _ New StreamWriter(filename, False, Encoding.ASCII) Directly creating the StreamWriter gives us more flexibility in how the StreamWriter is configured. The second parameter is set to False and indicates to Visit the Agilent Developer Network at http://www.agilent.com/find/ADN

Agilent Developer Network White Paper 6 of 17 Storing Test Data to File Using Microsoft® Visual Basic® .NET November 2002 the StreamWriter constructor that we want to create a new file instead of appending to an existing file. The third parameter (Encoding.ASCII) results in the test data being saved using ASCII encoding.

Saving test data out in name value format is pretty easy. You just write out a name for each element of the test data followed by an “=” sign followed by the value for that element of the test data. The file contents produced by this approach are shown in Figure 8.

Figure 8 Name Value Pair File Contents DutId=A00100 Timestamp=10/11/2002 2:52:26 AM StationId=1 Data=1.1,2.2,3.3 The code to read the test data from this file is shown in Figure 9. Figure 9 Reading from a Name Value Pair Text File ' Example 2b in the sample project Dim dutId As String Dim timestamp As DateTime Dim stationId As Integer Dim data() As Double ' Open file for reading as text Dim reader As StreamReader = New StreamReader(filename) ' Read the file one line at a time Dim line As String While reader.Peek -1 line = reader.ReadLine() ' Extract name/value pairs Dim ndx As Integer = line.IndexOf("=") If ndx -1 Then Dim name As String = line.Substring(0, ndx) Dim value As String = line.Substring(ndx + 1) ' Parse each value - note with this scheme you don't ' have to read the data in the order it was written Select Case name Case "Timestamp" timestamp = DateTime.Parse(value) Case "StationId" stationId = Integer.Parse(value) Case "Data" Dim dataStr() As String = value.Split(",") data = New Double(UBound(dataStr)) {} Dim i As Integer For i = 0 To UBound(dataStr) data(i) = Double.Parse(dataStr(i)) Next Visit the Agilent Developer Network at http://www.agilent.com/find/ADN

Agilent Developer Network White Paper 7 of 17 Storing Test Data to File Using Microsoft® Visual Basic® .NET November 2002 Case "DutId" dutId = value End Select End If End While ' Close StreamReader when done reader.Close() Again, this example directly creates a StreamReader but this time rather than reading in the entire file contents at once using the StreamReader.ReadToEnd method, it reads in the file one line at a time using the StreamReader.ReadLine method. This works nicely because each name value pair was saved on its own line. The example uses a Select statement to execute different parse code depending upon the name value pair that was read from each line of the file.

One minor advantage of this approach is that the data doesn’t have to appear in any particular order in the file. Putting the code in the Case statements is manageable for a small example like this but if the amount of data starts getting larger you might want to put the parsing code for each piece of data in its own function. Saving Data Using BinaryWriter The previous two examples create files that are human readable which has it advantages. However, if you have a large amount of test data, reading in and parsing data stored as text can be slow. If performance is your primary concern then consider saving your test data to a binary file as shown in Figure 10.

Figure 10 Writing to a File Using BinaryWriter ' Example 3a in the sample project ' Create test data Dim dutId As String = "A00102" Dim timestamp As DateTime = DateTime.Now Dim stationId As Integer = 5 Dim data() As Double = New Double() {3.3, 4.4, 5.5} ' Create FileStream for writing binary data ' This locks the file (FileShare.None) until FileStream is closed Dim stream As FileStream = New FileStream(filename, FileMode.Create, _ FileAccess.Write, FileShare.None) ' Create BinaryWriter on top of stream to write data in binary format Dim writer As BinaryWriter = New BinaryWriter(stream) ' Write data in binary format writer.Write(dutId) writer.Write(timestamp) writer.Write(stationId) writer.Write(UBound(data)) Dim ndx As Integer For ndx = 0 To UBound(data) writer.Write(data(ndx)) Next Visit the Agilent Developer Network at http://www.agilent.com/find/ADN

Agilent Developer Network White Paper 8 of 17 Storing Test Data to File Using Microsoft® Visual Basic® .NET November 2002 ' Close BinaryWriter which automatically closes underlying FileStream writer.Close() In this example, a file is opened in yet another way using the System.IO.FileStream class. So far our examples have opened a file using the following classes: File, StreamWriter and now FileStream. What’s going on? The class that actually does file I/O is FileStream. FileStream derives from the base class Stream. Stream defines a common set of methods for reading and writing bytes, determining the length of a stream and setting the seek position.

Stream itself cannot be created but its derived classes can. These include BufferedStream, MemoryStream, NetworkStream, CryptoStream and of course FileStream. Now you could do all of your file I/O using just the FileStream class. The many different FileStream constructors allow you to select Read/Write/ReadWrite mode, Open/Create/Append/Truncate mode, file sharing options, buffer size and asynchronous mode. This gives you a lot of flexibility however reading and writing data to a file in byte format is not the easiest way to save your test data. Microsoft recognized this and provided several different reader and writer classes.

StreamReader and StreamWriter work with any type of stream object including FileStream and provide services for reading and writing data in text format. StreamWriter allows you to select the text encoding you desire; remember it defaults to UTF-8. StreamReader can typically determine the encoding type based on the existence and value of a byte order mark (BOM) in the text file that it is reading. Of course, you can always explicitly choose which encoding that StreamReader should use.

As we saw in Figure 7 and Figure 9, both StreamWriter and StreamReader provide constructors that take a filename. Internally, these classes create a FileStream object given a filename. In Figure 4, the File.CreateText method creates a StreamWriter that internally creates a FileStream object to write onto. You may also notice the StringReader and StringWriter classes, which are very similar in functionality to a StreamReader and StreamWriter. The primary difference is that StringReader and StringWriter operate on a string instead of a Stream.

BinaryReader and BinaryWriter provide services for reading and writing primitive data types in binary format.

As you might notice in Figure 10, writing data to file using the BinaryWriter is very straightforward. One thing to note about all Stream related reader and writer objects is that when you call the Close method on them, they automatically close the stream they are using. The contents of the generated file are shown in Figure 11. Because it is a binary format, characters that cannot be displayed are represented by their 3 digit decimal value.

Figure 11 Binary File Contents 006 A 0 0 1 0 2 022 1 0 / 1 2 / 2 0 0 2 1 1 : 2 5 : 2 0 P M 005 000 000 000 002 000 000 000 f f f f f f 010 @ 017 @ 000 000 000 000 000 000 022 @ Visit the Agilent Developer Network at http://www.agilent.com/find/ADN

Agilent Developer Network White Paper 9 of 17 Storing Test Data to File Using Microsoft® Visual Basic® .NET November 2002 The code to read the test data from this binary file is shown in Figure 12. Figure 12 Reading from a File Using BinaryReader ' Example 3b in the sample project Dim dutId As String Dim timestamp As DateTime Dim stationId As Integer Dim data() As Double ' Open FileStream for reading binary data Dim stream As FileStream = New FileStream(filename, FileMode.Open) ' Create BinaryReader on top of stream to read data in binary format Dim reader As BinaryReader = New BinaryReader(stream) ' Read data in same order it was written dutId = reader.ReadString() timestamp = DateTime.Parse(reader.ReadString()) stationId = reader.ReadInt32() Dim upperBound As Integer = reader.ReadInt32() data = New Double(upperBound) {} Dim i As Integer For i = 0 To upperBound data(i) = reader.ReadDouble() Next ' Close BinaryReader which automatically closes underlying FileStream reader.Close() Reading the test data from the binary file is also pretty straightforward.

After creating a FileStream and a BinaryReader, this example uses BinaryReader methods like ReadString, ReadInt32 and ReadDouble to read the test data. One thing to note is that BinaryReader and BinaryWriter don’t handle DateTime objects directly so you have to store the DateTime object as text and then read it back in as text. Like the first example (Figure 6) you have to read the test data in the same order in which it was saved out.

Saving Data Using Binary Serialization You have seen various different ways to save test data in both text and binary format. One thing in common with all of these approaches is that code had to be written to save each individual element of the test data as well as parse each individual element when reading the data back from file. Fortunately, the FCL provides a mechanism referred to as serialization that does all of this tedious work for us. All you have to do is package the test data in a class and mark that class as serializable as shown in Figure 13. The FCL introduces a new concept called attributes.

Just decorate the TestData class with the SerializableAttribute using the notation “”. This attribute tells the CLR that any objects of type TestData can be serialized.

Visit the Agilent Developer Network at http://www.agilent.com/find/ADN