We got a few emails over our saying that "users really don't care that much about speed any more" when we were making points for Flash development.
In context we stand by the statement.
Don't get us wrong, we've loved ASP.Net since before the betas, it's a natural and intuitive technology for a long-time VB person. However, ASP.Net is just what its name implies, it is a Serverside technology. If speed were the absolute measure of a good end-user application then so many developers wouldn't be drag & dropping so many ASP.Net controls on so many pages and calling the job "Done", letting every single control postback to the server and roundtrip 99% unchanged information - and worse, hitting a database with each trip for nothing new. If perceived responsiveness alone were the definition of a good program then a heck of a lot more hand-tooled Javascript would be being put into all of those ASPXes.
Does that mean that these applications are bad? Not if the user is getting what they need.
Users are used to the web and they're used to pages taking time and they are used to having to enter information into 20+ textboxes only to be presented with the same page over and over again forcing them to re-enter mistaken values one value/reload at a time. Our bin code may be tight and "compiled" but for an Internet app, there's a long and slow path between the server and the user's screen, especially at the still predominant 56K. Actionscript isn't compiled but the GUI and a good chunk of validation logic is sitting on the client in a correctly written Flash "Rich App", meaning fewer wasteful round trips required to give the user a speedy and responsive experience (so long as the GUI doesn't get too "Flashy"). Compiled is not automatically better than interpreted if compiled passes more bloat over the wire.
The point was and is that what makes a good app for end users isn't always the theoretical stuff they teach in Computer Science class. BUT... we aren't implying that anyone should give up trying to give a better than average performance if time and resources allow a bit of tweaking.
For example, it's easy to write a recursive file search routine, right? Even if you've never done it in .Net or haven't done it in a while, you only have to know a few obvious methods from System.IO: [DirectoryInfo/Directory].GetDirectories, [DirectoryInfo/Directory].GetFiles and [FileInfo/File].[File properties].
Option Strict On
Module NetVersion
Private m_iFileCount As Integer
Private m_ext As String = "*.*"
Function SearchNet(ByVal StartDir As String, ByVal Ext As String) _
As Integer
m_iFileCount = 0
m_ext = Ext
DoSearch(StartDir)
Return m_iFileCount
End Function
Private Sub DoSearch(ByVal currentPath As String)
'called recursively
'we mix and match Directory, DirectoryInfo and FileInfo
'to get strings or objects where most useful
Dim arFilesInDir() As IO.FileInfo
Dim arDirsInDir() As String
Dim oFile As IO.FileInfo
Dim sDir As String
Dim oDir As IO.DirectoryInfo
Try
arDirsInDir = IO.Directory.GetDirectories(currentPath)
oDir = New IO.DirectoryInfo(currentPath)
arFilesInDir = oDir.GetFiles(m_ext)
'FileInfos are used because one object will
'expose all the various properties (size, dates, etc.),
'this provides apples to apples comparison
'to the API version
For Each oFile In arFilesInDir
m_iFileCount += 1
Next
''comment out the above and
''uncomment the following to see if
''not iterating through FileInfos will help speed...
'm_iFileCount += arFilesInDir.Length
''... in my testing it made no difference at all
For Each sDir In arDirsInDir
DoSearch(sDir)
Next
If Not oFile Is Nothing Then
oFile = Nothing
End If
If Not oDir Is Nothing Then
oDir = Nothing
End If
Catch ex As UnauthorizedAccessException
'you hit a protected file/folder
'let it go at runtime
'this could be avoided by checking
'the IO.FileAtrributes for the oDir or oFile
'in the loop
Debug.WriteLine(ex.Message)
Catch ex As Exception
'Todo: remove to trim overhead
Debug.WriteLine(ex.ToString)
End Try
arFilesInDir = Nothing
arDirsInDir = Nothing
End Sub
End Module
73 lines including whitespace and comments. It's simple. It works. It gets the job done.
Contrast that with the style that we prefer to use in our VB7 apps:
Imports System.Runtime.InteropServices
Module APIVersion
'code adapted to VB.Net from VB6 sample at:
' http://support.microsoft.com/default.aspx/kb/185476/EN-US/
Private m_iFileCount As Integer
Private Const INVALID_HANDLE_VALUE As Integer = -1
Private Const FILE_ATTRIBUTE_DIRECTORY As Integer = &H10
<StructLayout(LayoutKind.Sequential, Pack:=1, CharSet:=CharSet.Auto)> _
Structure WIN_FIND_DATA
Dim fileAttributes As Integer
Dim creationTime As Long
Dim lastAccessTime As Long
Dim lastWriteTime As Long
Dim nFileSizeHigh As Integer
Dim nFileSizeLow As Integer
Dim dwReserved0 As Integer
Dim dwReserved1 As Integer
<MarshalAs(UnmanagedType.ByValTStr, SizeConst:=260)> _
Dim fileName As String
<MarshalAs(UnmanagedType.ByValTStr, SizeConst:=14)> _
Dim alternateFileName As String
End Structure 'WIN_FIND_DATA
<StructLayout(LayoutKind.Sequential, Pack:=1, CharSet:=CharSet.Auto)> _
Structure FILE_PARAMS
Dim bRecurse As Boolean
Dim bList As Boolean
Dim bFound As Boolean
Dim sFileRoot As String
Dim sFileNameExt As String
Dim sResult As String
Dim nFileCount As Integer
Dim nFileSize As Double
End Structure
Declare Auto Function FindFirstFile Lib "kernel32.dll" _
(<MarshalAs(UnmanagedType.LPTStr)> ByVal lpFileName As String, _
<Out()> ByRef lpFindFileData As WIN_FIND_DATA) As IntPtr
Declare Auto Function FindNextFile Lib "kernel32.dll" _
(ByVal hFindFile As IntPtr, <Out()> _
ByRef lpFindFileData As WIN_FIND_DATA) As IntPtr
Declare Auto Function FindClose Lib "kernel32.dll" _
(ByVal hFindFile As IntPtr) As IntPtr
Function SearchAPI(ByVal StartDir As String, ByVal Ext As String) _
As Integer
m_iFileCount = 0
Dim FP As FILE_PARAMS
With FP
.sFileRoot = StartDir
.sFileNameExt = Ext
.bRecurse = True
End With
DoSearch(FP)
Return m_iFileCount
End Function
Private Function DoSearch(ByVal FP As FILE_PARAMS) As Double
'local working variables
Dim WFD As WIN_FIND_DATA
Dim hFile As IntPtr
Dim nSize As Integer
Dim sPath As String
Dim sRoot As String
Dim sTmp As String
sRoot = QualifyPath(FP.sFileRoot)
sPath = sRoot & "*.*"
'obtain handle to the first match
hFile = FindFirstFile(sPath, WFD)
'if valid ...
If hFile.ToInt32 <> INVALID_HANDLE_VALUE Then
'This is where the method obtains the file
'list and data for the folder passed.
GetFileInformation(FP)
Do
'if the returned item is a folder...
If (WFD.fileAttributes And FILE_ATTRIBUTE_DIRECTORY) = _
FILE_ATTRIBUTE_DIRECTORY Then
'..and the Recurse flag was specified
If FP.bRecurse Then
'remove trailing nulls
sTmp = TrimNull(WFD.fileName)
'and if the folder is not the default
'self and parent folders...
If sTmp <> "." And sTmp <> ".." Then
'..then the item is a real folder, which
'may contain other sub folders, so assign
'the new folder name to FP.sFileRoot and
'recursively call this function again with
'the amended information.
FP.sFileRoot = sRoot & sTmp
DoSearch(FP)
End If 'sTmp
End If 'brecurse
End If
'continue looping until FindNextFile returns
'no more matches
Loop While FindNextFile(hFile, WFD).ToInt32 <> 0
'close the find handle
hFile = FindClose(hFile)
End If
End Function
Private Function GetFileInformation(ByVal FP As FILE_PARAMS) As Long
'local working variables
Dim WFD As WIN_FIND_DATA
Dim hFile As IntPtr
Dim sPath As String
Dim sRoot As String
Dim sTmp As String
'FP.sFileRoot (assigned to sRoot) contains
'the path to search.
'
'FP.sFileNameExt (assigned to sPath) contains
'the full path and filespec.
sRoot = QualifyPath(FP.sFileRoot)
sPath = sRoot & FP.sFileNameExt
'obtain handle to the first filespec match
hFile = FindFirstFile(sPath, WFD)
'if valid ...
If hFile.ToInt32 <> INVALID_HANDLE_VALUE Then
Do
'remove trailing nulls
sTmp = TrimNull(WFD.fileName)
'Even though this routine uses filespecs,
'*.* is still valid and will cause the search
'to return folders as well as files, so a
'check against folders is still required.
If Not (WFD.fileAttributes And FILE_ATTRIBUTE_DIRECTORY) _
= FILE_ATTRIBUTE_DIRECTORY Then
'here's where you'd raise an event with file
'properties from the WinFindData struct
m_iFileCount += 1
End If
Loop While FindNextFile(hFile, WFD).ToInt32 <> 0
'close the handle
hFile = FindClose(hFile)
End If
End Function
Public Function TrimNull(ByVal startstr As String) As String
'classic helper
'returns the string up to the first
'null, if present, or the passed string
Dim pos As Integer
pos = InStr(startstr, Chr(0))
If pos <> 0 Then
TrimNull = Left$(startstr, pos - 1)
Exit Function
End If
TrimNull = startstr
End Function
Private Function QualifyPath(ByVal sPath As String) As String
'helper
'assures that a passed path ends in a slash
If Right$(sPath, 1) <> "\" Then
QualifyPath = sPath & "\"
Else : QualifyPath = sPath
End If
End Function
End Module
Ohhhh-Kay. Raise your hand if you can type that off the top of your head, even after doing it a few times letter by letter. 192 lines as written, if you delete all of the comments it drops to 150 but still that's more than double the .Net version and without comments it's not something you'd like to come across while debugging at 3AM on a Monday before release.
So why on earth would we prefer it over the simple .Net version? Because...
Start a new VB exe project and copy both of the above code areas to modules (NetVersion.vb and APIVersion.vb). Now on the default form add three buttons (butGetSource, butDoNet and butDoAPI), a textbox (txtExt), and three borderstyle=FixedSingle labels (lblSourceFolder, lblNetResult and lblAPIResult). Here's mine, you should be able to figure what's what:
Switch to code view on the form and paste in the calls:
Private m_SourceFolder As String = ""
Private Sub butGetSource_Click(ByVal sender As System.Object, _
ByVal e As System.EventArgs) Handles butGetSource.Click
Dim fo As New FolderBrowserDialog
With fo
If .ShowDialog = DialogResult.OK Then
m_SourceFOlder = .SelectedPath
lblSourceFolder.Text = m_SourceFOlder
End If
End With
If Not fo Is Nothing Then
fo.Dispose()
fo = Nothing
End If
End Sub
Private Sub butDoNet_Click(ByVal sender As System.Object, _
ByVal e As System.EventArgs) Handles butDoNet.Click
'do recursive file search with .Net
lblNetResult.Text = "running..."
Me.Refresh()
Dim iStart As Integer = Environment.TickCount
Dim iEnd As Integer
Dim iFileCount As Integer = SearchNet(m_SourceFolder, txtExt.Text)
iEnd = Environment.TickCount
lblNetResult.Text = "ms: " & (iEnd - iStart).ToString & _
" (" & iFileCount.ToString & ")"
End Sub
Private Sub butDoAPI_Click(ByVal sender As System.Object, _
ByVal e As System.EventArgs) Handles butDoAPI.Click
'do recursive file search with API
lblAPIResult.Text = "running..."
Me.Refresh()
Dim iStart As Integer = Environment.TickCount
Dim iEnd As Integer
Dim iFileCount As Integer = SearchAPI(m_SourceFolder, txtExt.Text)
iEnd = Environment.TickCount
lblAPIResult.Text = "ms: " & (iEnd - iStart).ToString & _
" (" & iFileCount.ToString & ")"
End Sub
Compile and run (as with all .Net code, you want to run the methods at least once just to get them jitted, so ignore the results of your first button clicks). Hardware plays a big part and a purist will say that a PerformanceCounter should be used for "real" timing, but the precision of TickCount is good enough to show the general idea in this case.
If you don't have time to do this yourself, take a look at our typical results (the dev machine is a Toshiba P25 2.8HT with 1.5GB ram):
Same result (142261 files), but a major and consistant difference in speed.
With hard drives, dvd/Rs and flashdrives getting bigger and bigger a file search is one place where going low to gain performance is worth every single one of those extra old fashioned non-portable lines.
If you're just thinking of copying the core code into your app, we suggest that you put it into a class and call it in a new thread so that the user can move the form around, have it raise events back to the GUI so they get cues that the process is working (and so you can trap the file information), and you should provide a ByRef Cancel in the events so the user is given a chance to bail out. We didn't add that real-app stuff here so we could focus on the meat of the matter.
And that meat is: Sometimes the biggest performance killers are out of your control. The key, always, is to find the bottleneck and if you can't fix it then find a way, even a bizzare way that a trained (or habitual) nerd would laugh at, to minimize it. This applies to using Win32API calls on file searches even though .Net has those easy three methods that "everyone uses" in System.IO ... and it applies to considering Flash for web apps, because even though it's interpreted it can be "faster" due to it's using real client-side code and lean data-passing instead of bloated round-trips just for UI minutia.
Sometimes it's book learnin', sometimes it's street logic. Always it's what's best for your users.
Hope it helps!
Robert Smith
Kirkland, WA
added to smithvoice february 2005
for our own copying and pasting, here's an adaptation that accepts an array of filespecs:
Option Strict On
Imports System.Runtime.InteropServices
Module APIFilespecs
'code adapted to VB.Net from VB6 sample at: from:
' http://support.microsoft.com/default.aspx/kb/185476/EN-US/
Private m_iFileCount As Integer
Private m_arFileSpecs() As String
Private Const INVALID_HANDLE_VALUE As Integer = -1
Private Const FILE_ATTRIBUTE_DIRECTORY As Integer = &H10
<StructLayout(LayoutKind.Sequential, Pack:=1, CharSet:=CharSet.Auto)> _
Structure FindData
Dim fileAttributes As Integer
Dim creationTime As Long
Dim lastAccessTime As Long
Dim lastWriteTime As Long
Dim nFileSizeHigh As Integer
Dim nFileSizeLow As Integer
Dim dwReserved0 As Integer
Dim dwReserved1 As Integer
<MarshalAs(UnmanagedType.ByValTStr, SizeConst:=260)> _
Dim fileName As String
<MarshalAs(UnmanagedType.ByValTStr, SizeConst:=14)> _
Dim alternateFileName As String
End Structure 'FindData
<StructLayout(LayoutKind.Sequential, Pack:=1, CharSet:=CharSet.Auto)> _
Structure FILE_PARAMS
Dim bRecurse As Boolean
Dim bList As Boolean
Dim bFound As Boolean
Dim sFileRoot As String
Dim sFileNameExt As String
Dim sResult As String
Dim nFileCount As Integer
Dim nFileSize As Double
End Structure
Declare Auto Function FindFirstFile Lib "kernel32.dll" _
(<MarshalAs(UnmanagedType.LPTStr)> ByVal lpFileName As String, _
<Out()> ByRef lpFindFileData As FindData) As IntPtr
Declare Auto Function FindNextFile Lib "kernel32.dll" _
(ByVal hFindFile As IntPtr, _
<Out()> ByRef lpFindFileData As FindData) As IntPtr
Declare Auto Function FindClose Lib "kernel32.dll" _
(ByVal hFindFile As IntPtr) As IntPtr
Function SearchAPIFileSpec(ByVal StartDir As String, _
ByVal fileSpec() As String) As Integer
m_iFileCount = 0
m_arFileSpecs = fileSpec
Dim FP As FILE_PARAMS
With FP
.sFileRoot = StartDir
.bRecurse = True
End With
DoSearch(FP)
Return m_iFileCount
End Function
Private Function DoSearch(ByVal FP As FILE_PARAMS) As Double
'local working variables
Dim WFD As FindData
Dim hFile As IntPtr
Dim nSize As Integer
Dim sPath As String
Dim sRoot As String
Dim sTmp As String
sRoot = QualifyPath(FP.sFileRoot)
sPath = sRoot & "*.*"
'obtain handle to the first match
hFile = FindFirstFile(sPath, WFD)
'if valid ...
If hFile.ToInt32 <> INVALID_HANDLE_VALUE Then
'This is where the method obtains the file
'list and data for the folder passed.
For i As Integer = 0 To m_arFileSpecs.Length - 1
GetFileInformation(FP, m_arFileSpecs(i))
Next
Do
'if the returned item is a folder...
If (WFD.fileAttributes And FILE_ATTRIBUTE_DIRECTORY) = _
FILE_ATTRIBUTE_DIRECTORY Then
'..and the Recurse flag was specified
If FP.bRecurse Then
'remove trailing nulls
sTmp = TrimNull(WFD.fileName)
'and if the folder is not the default
'self and parent folders...
If sTmp <> "." And sTmp <> ".." Then
'..then the item is a real folder, which
'may contain other sub folders, so assign
'the new folder name to FP.sFileRoot and
'recursively call this function again with
'the amended information.
FP.sFileRoot = sRoot & sTmp
DoSearch(FP)
End If 'sTmp
End If 'brecurse
End If
'continue looping until FindNextFile returns no more matches
Loop While FindNextFile(hFile, WFD).ToInt32 <> 0
'close the find handle
hFile = FindClose(hFile)
End If
End Function
Private Function GetFileInformation(ByVal FP As FILE_PARAMS,
ByVal fileSpec As String) As Long
'local working variables
Dim WFD As FindData
Dim hFile As IntPtr
Dim sPath As String
Dim sRoot As String
Dim sTmp As String
'FP.sFileRoot (assigned to sRoot) contains
'the path to search.
'
'FP.sFileNameExt (assigned to sPath) contains
'the full path and filespec.
sRoot = QualifyPath(FP.sFileRoot)
FP.sFileNameExt = fileSpec
sPath = sRoot & FP.sFileNameExt
'obtain handle to the first filespec match
hFile = FindFirstFile(sPath, WFD)
'if valid ...
If hFile.ToInt32 <> INVALID_HANDLE_VALUE Then
Do
'remove trailing nulls
sTmp = TrimNull(WFD.fileName)
'Even though this routine uses filespecs,
'*.* is still valid and will cause the search
'to return folders as well as files, so a
'check against folders is still required.
If Not (WFD.fileAttributes And FILE_ATTRIBUTE_DIRECTORY) _
= FILE_ATTRIBUTE_DIRECTORY Then
m_iFileCount += 1
'test only, putting to Output influences the timing
'note the use of lastWriteTime because
'creationTime returns the FOLDER date, not the file
'Debug.WriteLine(WFD.fileName & vbTab & _
' Date.FromFileTime(WFD.lastWriteTime).ToString)
End If
Loop While FindNextFile(hFile, WFD).ToInt32 <> 0
'close the handle
hFile = FindClose(hFile)
End If
End Function
Public Function TrimNull(ByVal startstr As String) As String
'classic helper
'returns the string up to the first
'null, if present, or the passed string
Dim pos As Integer
pos = InStr(startstr, Chr(0))
If pos <> 0 Then
TrimNull = Left$(startstr, pos - 1)
Exit Function
End If
TrimNull = startstr
End Function
Private Function QualifyPath(ByVal sPath As String) As String
'helper
'assures that a passed path ends in a slash
If Right$(sPath, 1) <> "\" Then
QualifyPath = sPath & "\"
Else : QualifyPath = sPath
End If
End Function
End Module
... and a sample of how to call it (building off of the above test harness):
Private Sub butDoAPIFS_Click(ByVal sender As System.Object, _
ByVal e As System.EventArgs) Handles butDoAPIFS.Click
'do recursive file search with API
lblAPIResult.Text = "running..."
Me.Refresh()
'build the array with the UI, this hardcode is just for example
Dim arFS() As String = {"*.jpg", "*.mpg"}
Dim iStart As Integer = Environment.TickCount
Dim iEnd As Integer
Dim iFileCount As Integer = SearchAPIFileSpec(m_SourceFolder, arFS)
iEnd = Environment.TickCount
lblAPIResultFS.Text = "ms: " & (iEnd - iStart).ToString & _
" (" & iFileCount.ToString & ")"
End Sub