Analytics


Google

Wednesday, April 9, 2008

Off line scalable computing

For our annual dinner, we need to create a windows application for registering employees as they enter the hall to be used to perform lucky draw. Since there is a large number of employees, we need to be able scale to multiple computers for scanning their badges.

We also want to simplify the requirement for the hardware so we do not want to install databases etc on the computer. We just want to have .Net Framework and our application on the computers. So I thought I will share an approach with you.

We assume that we will have a computers together and then use shared folders.

The following is a snapshot of the demo program:



If you key in input into one screen, it will be captured into the second screen. Each screen represents a different computer. In this example I used two folders.

This is the config file for the first computer:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<appSettings>
<add key="ConName" value="DT01" />
<add key="monDir" value="c:\mon\drv1\" />
<add key="dirCnt" value="2" />
<add key="dir1" value="c:\mon\drv2\" />
<add key="dir2" value="c:\mon\drv3\" />
</appSettings>
</configuration>

This is the config file for the second computer:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<appSettings>
<add key="ConName" value="DT02" />
<add key="monDir" value="c:\mon\drv2\" />
<add key="dirCnt" value="2" />
<add key="dir1" value="c:\mon\drv1\" />
<add key="dir2" value="c:\mon\drv3\" />
</appSettings>
</configuration>


In this case, each time we scan a badge in each computer, it will publish a text file to the folders in for the other computers (number of computers identified by the dirCnt. The application will also monitor any new files published by other computers to it - identified by monDir. ConName is the console name which identifies the computer name and is also used to create distinct file name when publishing.

For the sake of demo, I create two listbox. In order to support threadsafe updating of Windows Control I created a delegate (This was discussed in an earlier blog).

Delegate Sub SetTextCallback(ByVal [text] As String)

I have created two subroutine one for each listbox:


' Used to allow multiple threads to update the textbox.
Private Sub SetText(ByVal [text] As String)
' InvokeRequired required compares the thread ID of the
' calling thread to the thread ID of the creating thread.
' If these threads are different, it returns true.
If Me.lbMsg.InvokeRequired Then
Dim d As New SetTextCallback(AddressOf SetText)
Me.Invoke(d, New Object() {[text]})
Else
Me.lbMsg.Items.Add([text])
chkLBSize()
End If
End Sub

' Used to allow multiple threads to update the textbox.
Private Sub SetText2(ByVal [text] As String)
' InvokeRequired required compares the thread ID of the
' calling thread to the thread ID of the creating thread.
' If these threads are different, it returns true.
If Me.lbMsg2.InvokeRequired Then
Dim d As New SetTextCallback(AddressOf SetText2)
Me.Invoke(d, New Object() {[text]})
Else
Me.lbMsg2.Items.Add([text])
chkLBSize()
End If
End Sub


' To maintain the entries to lex then maxCnt
Private Sub chkLBSize()
Dim loopCnt, indx As Integer

If lbMsg.Items.Count > maxLen Then
indx = 0
For loopCnt = 1 To delNum
lbMsg.Items.RemoveAt(indx)
If lbMsg.Items.Count <= 0 Then
Exit For
End If
Next
End If

If lbMsg2.Items.Count > maxLen Then
indx = 0
For loopCnt = 1 To delNum
lbMsg2.Items.RemoveAt(indx)
If lbMsg2.Items.Count <= 0 Then
Exit For
End If
Next
End If
End Sub

ChkLBSize controls the number of rows maintained in each ListBox.

Next we have a global variable to facilitate stopping of the folder monitoring:

Dim StopMon As Boolean = False

We can have also some constants to determine how many rows to maintain in the listbox and how many rows to remove once the threshold is exceeded:

Const maxLen As Integer = 20
Const delNum As Integer = 5

We then have a button to start and a button to stop the folder monitoring:


Private Sub btnWatch_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnWatch.Click
Dim createThrd As Boolean = True

If IsNothing(monThrd) = False Then
If monThrd.IsAlive Then
createThrd = False
End If
End If
If createThrd Then

Monitor.Enter(StopMon)
StopMon = False
Monitor.Exit(StopMon)

Dim thrd As New Thread(AddressOf monFile)
thrd.Start()
monThrd = thrd
Else
SetText(Now & " - Monitoring already started")
End If
End Sub


Private Sub btnStopWatch_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnStopWatch.Click
Monitor.Enter(StopMon)
StopMon = True
Monitor.Exit(StopMon)
End Sub


The monitoring process is as follows:
    Private Sub monFile()
Dim monDir As String = AppSettings("monDir")

SetText(String.Format("{0} - Monitor started", Now))

While (True)
If StopMon Then
Exit While
End If
processDir(monDir)
Thread.Sleep(600)
End While
SetText(String.Format("{0} - Monitor Stopped", Now))

End Sub

Sub processDir(ByVal pDir As String)
Dim fName, fName2 As String
Dim fObj As File

If (Directory.Exists(pDir) = False) Then
SetText(String.Format("{0} - {1} directory does not exist", Now, pDir))
Else
For Each fName In Directory.GetFiles(pDir)
fName2 = Path.GetFileName(fName)
SetText(String.Format("{0} - process file {1} - name is {2}", Now, fName, fName2))
processFile(fName, fName2, pDir)
Next
End If

End Sub

Sub processFile(ByVal pFullFname As String, ByVal pFname As String, ByVal monDir As String)
Dim sr As StreamReader = New StreamReader(pFullFname)
Dim rline, tFolder, tFile As String

rline = sr.ReadLine
SetText2(rline)
sr.Close()

tFolder = monDir & "done\"
tFile = tFolder & pFname
File.Move(pFullFname, tFile)
SetText(String.Format("{0} - processed completed. Moved {1} to {2}", Now, pFullFname, tFile))

End Sub


Notice that the process is broken into three parts - can be combined if you want to. I prefer to do it this way. Go through a infinite loop and exit only when the StopMon is set to True. IThe processDir will then look up all the files, extract out the parts of the filename and pass the processing to the processFile. In processFile, I will read the file and then perform all the necessary process. In this example, it just display the output to the lbMsg2 by call SetText2.

As for the publishing, there is a button to submit the input from the TextBox:

Private Sub btnSubmit_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnSubmit.Click

' Locks vEntry so that other threads cannot access
Monitor.Enter(vEntry)
vEntry = txtEntry.Text
' Unlock vEntry after use
Monitor.Exit(vEntry)
spawnThrd()

SetText2(txtEntry.Text)
End Sub


In this case, I use a global variable vEntry. Monitor is used to lock the string object when updating the value. This value is then used to write to the files. The following generates the files:
    Sub spawnThrd()
Dim thrd As New Thread(AddressOf creFile)
thrd.Start()
End Sub

Sub creFile()
Dim fCnt, vFCnt, loopCnt As Integer
Dim vMsg, fName, fWriteDir, fMoveDir, dirKey As String

Monitor.Enter(vEntry)
vMsg = vEntry
Monitor.Exit(vEntry)

Monitor.Enter(vFileCnt)
vFCnt = vFileCnt
vFileCnt += 1
Monitor.Exit(vFileCnt)

fCnt = AppSettings("dirCnt")

For loopCnt = 1 To fCnt
dirKey = "dir" & loopCnt
SetText("dirKey is " & dirKey)
fMoveDir = AppSettings(dirKey)
fWriteDir = fMoveDir & "temp\"
fName = AppSettings("conName") & "_" & Format(Now, "yyyyMMddHH24mm") & "_" & vFCnt & ".txt"
SetText(String.Format("{0} - move dir is {1} write dir is {2} and filename is {3}", Now, fMoveDir, fWriteDir, fName))
writeFile(fMoveDir, fWriteDir, fName, vEntry)
Next

End Sub

Sub writeFile(ByVal pMoveDir As String, ByVal pWriteDir As String, ByVal pFname As String, ByVal pMsg As String)
Dim fName, tFname As String

fName = pWriteDir & pFname
tFname = pMoveDir & pFname

Dim sw As StreamWriter = New StreamWriter(fName)
sw.Write(pMsg)
sw.Close()
File.Move(fName, tFname)

SetText(String.Format("{0} - Wrote {1} and moved to {2}", Now, fName, tFname))

End Sub

SpawnThrd will create and start a thread for each of entries. creFile will loop through each of the folders identified by the dirX (as in dir1, dir2 etc) in the config file. The dirCnt facilitates this. The unique file name is achieve using a combination of Console Name (conName in config file), date and time and also a running number. Also to reduce the possibility of read/write contention, the files are written in a temp sub-folder before being moved to the main folder.

No comments: