Tuesday, June 24, 2008

Migrating from IIS 5 to IIS 6 - manual way

A few years back I tried to migrate from IIS 5 to IIS 6 with the tools provided but failed miserably. It is probably because of how we structured our content. Most of the resources I found refers to IIS Migration Tool because it is the easiest way to migrate. However, it may not work for some of you.

Microsoft expects most of our pages to be placed in c:\inetpub\wwwroot but it provides you an option to map any folder you can find on the server. The methods illustrated here are only works with Windows 2003 (it does not work with Windows XP.

Assuming that you started a new Windows 2003 server, first thing is to copy the codes from old server. Next right click on the IIS as follows:

This will generate a XML file containing all the information needed to generate the website.

Edit the xml file and then find the section showing IIsWebVirtualDir as shown below and then duplicate it. Replace the mapping with the ons you need:

<IIsWebVirtualDir Location ="/LM/W3SVC/1/ROOT/engr"
AccessFlags="AccessRead | AccessScript"
DirBrowseFlags="DirBrowseShowDate | DirBrowseShowTime | DirBrowseShowSize | DirBrowseShowExtension | DirBrowseShowLongDate | EnableDefaultDoc"

In the case above, the virtual directory is engr, so you should be able access it via http://localhost/engr/... The physical folder in the above exmaple is e:\engr.

You can then edit and map the folder accordingly. Assuming you have configured the IIS to use Framework 1.1, then you will see the following (which defines the default .Net framework when you configure an application):
<IIsWebVirtualDir Location ="/LM/W3SVC/1/ROOT"
AccessFlags="AccessRead | AccessScript"
AuthFlags="AuthAnonymous | AuthNTLM"
HttpCustomHeaders="MicrosoftOfficeWebServer: 5.0_Pub
X-Powered-By: ASP.NET"

Virtual folders that needs to use a framework other then the default will need to specify more as in:

<IIsWebVirtualDir Location ="/LM/W3SVC/1/ROOT/Test2"
AccessFlags="AccessRead | AccessScript"
DirBrowseFlags="DirBrowseShowDate | DirBrowseShowTime | DirBrowseShowSize | DirBrowseShowExtension | DirBrowseShowLongDate | EnableDefaultDoc"

Once you have modified and finalized your file, you can then import it in by right clicking on the IIS and then select new and then create Web Site (from file).

This trick can also be done on the folder level just right clicking on a specific folder get the smaller file. You can then edit the smaller file and add other folders. This is much faster then manually creating individual virtual folders. Once complete then you can use the above to create New Virtual Directory (from file).

Saturday, June 21, 2008

Improving performance of asp classic menu

I am sure there are still people out there that are still using classic asp even though has been out for several years and have gone through several versions.

The following are some tricks and tools I created to speed up asp classic.

Getting options from the menu will dramatically slow down the web page. This is because each time the page redraws, the server has to access the database to access the database and retrieve the information. One way is to have a console program (at the time, I did it in Java to pull the data and save it to a text file). Once, you have it in the text file you can then retrieve it using "File System Object".

To make it easier to maintain, I add an entry in the global.asa to point to the file. As in:

Application("procList") = "c:\extfile\proc.txt"

Your asp script can then check to see if the setting is available:

procFile = Application("procList")

if (procFile = "") then
response.write "Error Critical setup missing - step List"
end if

After that you can read that into an array, as shown below:

set fso = Server.CreateObject("Scripting.FileSystemObject")
redim p4Lookup(sizeInc)
p4Size = sizeInc
p4Max = 0
if (fso.FileExists(procFile)) then
set file = fso.OpenTextFile(procFile,1)
while file.AtEndOfStream <> True
tline = trim(file.readline())
if (tline <> "") then
p4Max = p4Max + 1
if (p4Max > p4Size) then
p4Size = p4Size + sizeInc
redim preserve p4Lookup(p4Size)
end if
p4Lookup(p4Max) = tline
end if
set fso = nothing
response.write "Error - critical file missing - prodList4"
end if

You can then create a function to draw the menu. You can then have create a subroutine and then place it between your html tags as follows:

<% ListArray p4Lookup, p4Max, "prod4", selprod4 %>

ListArray is the name of the subroutine, p4Lookup is the list read as shown above and p4Max is the number of items in the list. prod4 is the name of the menu item and selprod4 is the selected items.

To obtain the information, you will just do the following:

selprod4 = Request.form("prod4")

The list array subroutine is as follows:

' Create a form (pulldown) list from an Array

Sub ListArray(aList, aSize, SelName, SelRec)

'Parameters are as follows:
' aList - array of options
' aSize - number of items in array
' SelName - name of form selection input
' SelRec - Record selected


Response.Write "<select name=" & SelName & ">"
For i = 1 to aSize
Response.Write "<option "
if (aList(i) = SelRec) then
Response.Write "selected "
end if
Response.write "value=""" & _
aList(i) & """>"
Response.Write aList(i) & "</option>"
Response.Write "</select>"
End Sub

You can also have a script to allow multiple selection listbox as follows:
' Create Multiple Select list based on an array
' Parameters:
' aList Array List to use
' aLim Size of the array list
' selName Name for the multi select list
' selectedList Array containing the list of values selected
' selListSize Size of the selectedList
' Osize Size of the multiselect list
sub mSelList(aList, aLim, selName, selectedlist, selListSize, Osize)
response.write "<select name=" & selName & " size=" & Osize & " multiple>"
for i = 1 to aLim
response.write "<option"
if inList(selectedlist, selListSize, aList(i)) then
response.write " selected"
end if
response.write " value="""
response.write aList(i) & """>" & aList(i) & "</option>"
response.write "</select>"
end sub

Friday, June 20, 2008

Careful when installing Firefox 3.0

If you are already using Firefox, you should first make a backup of your bookmarks. I just upgraded to Firefox 3.0 yesterday and all the bookmarks were not carried over.

This is surprising, since it carried over everything else (cookies, stored password etc). The easiest way to backup the bookmarks, is to click on bookmarks, select organize and then export to html. You can then import the bookmarks from the html file after you perform your upgrade.

I know there are a lot of articles that talk about the features, so I am not going to do that here. Instead, I will talk about some of the things that affected me.
  1. Some addons that work are removed (even though they still work), such as torrent-search bar. You need to go to their web site to reinstall.
  2. You can now easily click on the star next to the current url to bookmark it, so the arrow in version 2 has been removed. The arrow was useful if you want to reload the page without submitting your form. Now, you have to hit enter.
Overall, I find some pages to be faster but others to be slower. I also found that some pages that used to be able to display no longer works especially if you are using ieTab.

Thursday, June 19, 2008

Sometimes we just need to reboot the server

Our team develops a lot of web applications (asp and However, we have seen several occasions when our program stops working properly. We are currently working with Windows 2000, so we use IIs5Recycle to improve the stability of the server.

I thought I will share some of the problems we saw and the solutions:

1. Selected pages stop responding while others continue to respond.
- In this case, we found that one of our Oracle stored packages became invalid. Once we recompiled the invalid package, everything returned to normal.

2. All pages suddenly slowed down or only pages stopped responding.
- One of the pages hit a contention and chewed up all the cpu.
- Stopping and starting the web publishing service did not help.
- Solution is to simply kill the process.

3. We use awstat to collect compile usage statistics of our website and suddenly, we encounter that the page was sending incomplete/invalid header.
- I tried many things but finally a reboot just fixes it.

We have seen also many cases when the pages started misbehaving, invalid access etc. So word of advise, before jumping into conclusion that the problem is with the code, try the following sequence:
  1. See if is chewing up all your cpu. If so, just kill it and the web server should return to normal. However, a note of caution, this should be done only if your server is not under powered and 100% cpu is not a normal occurrence.
  2. Use the services and stop and restart "World Wide Web Publishing Service". It should be last service in the list (default listing).
  3. Last resort, reboot Windows.
Windows 2003 provides a more robust IIS and introduces the concept of application pools. Instead of processes, you will now see w3wp.exe processes. These are the worker processes and you will see at least one for each running pool - you may see more.

Still the concept is similar.

Saturday, June 14, 2008

Reducing your file size

When writing documents in Word either on a PDA and computer, the size is normally at least 1 to 2k even if it is just contains one sentence. A work around is to save the document is .rtf (rich text) when you do that you will find the document a fraction of the size but still maintaining all your formating.

Here is a comparison (the doc file is 9.1 k while the rtf file is only 2.4k):

Size Filename
9,129 21 Memos.doc
2,443 21 Memos.rtf

Now majority of the photographs are taken using digital camera or scanned into the computer. We then take this pictures to post into our blogs. Majority of the time the pictures are used for illustration and does not need to be so big. One thing that can be done is to use free tools like Picasa to reduce the size. In the following example, my original photo is 2048 x 1536 pixels. The size is 739k. By using Picasa, I exported to file to 320 x 240 pixels, the size is reduced to 29k. This will make it much easier to upload.

Size Filename
756,426 21012007123.jpg
29,539 21012007123.jpg

By using the combination of the two, it will facilitate uploading your stories to websites like Triond.

Thursday, June 12, 2008

Screen scrapping using .Net

The following article was quite helpful in performing screen scrapping.

Highlights are as follows:
  • Before you can start screen scrapping, you need to know what is retrieved and what is posted by the browser. A free tool available to facilitate this is Fiddler.
  • If the website is an application, then you need to retrieve the view state.
  • There are some things that you can use as constant:
    • agent which tells the web server, what browser you are. In my case I use "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Maxthon; .NET CLR 1.1.4322; .NET CLR 2.0.50727)"
    • content-type, in my case I use "application/x-www-form-urlencoded".
  • Most sites have cookies so you need to retrieve the cookie and post each time (since web application are typically stateless.
Here are some sample snippets:

Const contentType As String = "application/x-www-form-urlencoded"
Const usrAgent As String = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Maxthon; .NET CLR 1.1.4322; .NET CLR 2.0.50727)"

Dim cookieCntr As New CookieContainer
Dim cookieColl As New CookieCollection
Dim cCache As New CredentialCache
Dim vCookie As Cookie
Dim rqst As HttpWebRequest
Dim rsp As HttpWebResponse
Dim sReader As StreamReader
Dim sWriter As StreamWriter
Dim postData As String
Dim rspStr As String
Dim url As String

'=========== First Post

rqst = WebRequest.Create(url)

rqst.CookieContainer = cookieCntr
'KeepAlive Setting
rqst.KeepAlive = False
rqst.AllowAutoRedirect = True
rqst.UserAgent = usrAgent
rqst.Credentials = cCache
rsp = rqst.GetResponse

sReader = New StreamReader(rsp.GetResponseStream)
cookieCntr = rqst.CookieContainer
rspStr = sReader.ReadToEnd
cCache = rqst.Credentials


'================ second post

rqst = WebRequest.Create(url)
rqst.AllowAutoRedirect = True
rqst.CookieContainer = cookieCntr
rqst.ContentType = contentType
rqst.Credentials = cCache
rqst.UserAgent = usrAgent
rqst.Method = "POST"

' Assign the postData information here

sWriter = New StreamWriter(rqst.GetRequestStream)

rsp = rqst.GetResponse
cCache = rqst.Credentials
rsp.Cookies = cookieColl
sReader = New StreamReader(rsp.GetResponseStream)
rspStr = sReader.ReadToEnd


Noticed that all the credentials, cookies that are collected during the first post will need to be sent back in subsequent post.

This is especially true if the web site requires log in. In the above example, what we process are just strings. If there are images or other objects, then they need to be processed separately. If you use Fiddler, you will find that there are separate stream to obtain separate objects from the server. If you have 10 images, you will get 10 additional streams. That is also the reason a lot of sites are using CAPTCHA to prevent automated screen scrapping.

Monday, June 9, 2008

Implication of Connection Pooling in Windows

When we write an application to run in Windows either using ASP or Windows Application; Windows will automatically perform connection pooling. I recently wrote an application that uses multi threads to scan the database using different criteria and then update the database.

One of the threads encountered the following error ORA-01422: exact fetch returns more than requested number of rows, due to the try and catch, the routine will exit and close the connection. However, I found that other threads apparently took over the session and encountered the following error instead: ORA-02067: transaction or savepoint rollback required.

Sunday, June 1, 2008

Optimizing Content in Quickr and Lotus NOTES

If you are using Lotus NOTES, you will find something interesting. If you copy and paste an image sometimes, you will find the document really big. The solution to that is to use paste special and then either use Device Independent BitMap or BitMap instead of RichText.

You will find the document to be significantly smaller. If you paste in RichText, the document may be > 2 Mbytes but by using paste special, the same document could be just a few hundred Kbyte.

Another thing we discovered, is you paste your content directly into Quickr document, opening such document subsequently will take several minutes. We have seen cases where it takes more than 10 minutes.

The solution is paste the content to a notepad and then copy from the notepad prior to pasting it into Quickr. Again this is probably linked to RichText.