Over 21,000 views on my Tumi TSA Lock video

Did you know how many people forget the number combination of their luggage locks? Imagine the trouble, if you just arrived in your hotel after a long flight.

In this short instruction video I posted a while back on youtube, I show how to open a Tumi TSA approved lock if you've forgotten the number combination.

And I just discovered it's been watched more than 21,000 times in the past 9 months. Wow ;-).

Amazing experience working with IBM China

I have very good memories from working with people in the AsiaPac region, both as a guest speaker at it-security (BlackHat) conferences in Hong Kong and in performing some forensics work for the Internal Security Department in Singapore.

This week I had my first experience in working together with team members from IBM in China. In one word: amazing! Let me tell you why:

Following the partnership that IBM and my company started this year (see this link >>), we needed to start supporting the non-western character-sets in our DNA solution in order to perform data collections on Lotus Domino servers in China.

For this purpose, Viktor Kranz (IBM's TF/DNA coordinator inside IBM) brought me in touch with Mr. Lin Tang of IBM China or 'Forest Soup' for those who get to know him ;-)

My encounters with Forest Soup:

• setting up a complete C++ dot Net 2003 IDE, including Notes C-api 
• understanding code logic for 9,000 LOC (C++): 25 minutes
• installation of a new Domino server: 5 minutes
• limit questions to just 4 sharp & smart ones: 5 minutes
• coming up with a plan
• daily status updates
• superb documentation
• clarity in communication
• entire new code done within a matter of days

Over the past 12.5 years in Trust Factory, I've met -and worked with- several uber-specialists such as Bill Buchan, Daniel Nashed, Richard van den Berg, Michael Dudding, Stephan Wissel, to name a few.

I'm ready to add another uber-specialist to my list: Amazing Mr Tang :-)

Nieuwe voorwaarden www.overheid.nl: Zijn jullie gek geworden!

Vanochtend vroeg was ik even benieuwd naar een paar kadastrale gegevens mbt. ons huis, en begreep dat ik op www.mijnoverheid.nl een overzicht kon inzien (zodat ik niet hiervoor hoef te betalen).

Tot mijn stomme verbazing wil de organisatie mij daarbij dwingen tot een keuze waar juridische consequenties aan kunnen kleven! Zie hieronder:

De volledige voorwaarden zijn hier te zien.

Als ik deze voorwaarden accepteer, ben ik dus verplicht om regelmatig hierop in te loggen. Doe ik dat niet, dan riskeer ik dat ik officiële communicatie van overheidsinstanties niet op tijd zie en mogelijk wettelijke termijnen overschrijd. Accepteer ik de voorwaarden niet, dan ben ik niet langer welkom om informatie over mezelf en mijn gezin in te zien op deze overheidsdienst.

ZIJN JULLIE NOU HELEMAAL GEK GEWORDEN DAAR?!

Update: onder protest de voorwaarden maar geaccepteerd en direct een reactieformulier ingevuld:


Bij het inloggen op deze dienst werd ik vanochtend geconfronteerd met een voldongen feit: danwel dwingende voorwaarden accepteren, waarbij ik mij verplicht tot het regelmatig inloggen en lezen van berichten op deze dienst, danwel mijn account opzeggen.

Volgens mij heb ik recht op het gebruik van deze -en hierop aangesloten diensten, zonder dat hieraan dergelijke verregaande verplichtingen en mogelijk juridische consequenties aan kleven.

Dergelijke praktijken stroken totaal niet met de doelstelling van deze dienst en passen totaal niet in het juridisch kader in Nederland.

Bij deze derhalve deel ik u dus mede dat ik de voorwaarden onder protest heb geaccepteerd, teneinde van de diensten gebruik te kunnen maken. Voor alle duidelijkheid meld ik u dat ik niet regelmatig zal inloggen, en dat u mijn berichtenbox niet voor communicatiedoeleinden mag gebruiken.

Graag zou ik zien dat u mijn inlog-gegevens actief houdt, waarbij u mijn -onder protest- gedane acceptatie van voorwaarden, in uw systemen en die van uw toeleveranciers weer ongedaan maakt.

Ik wacht uw reactie met belangstelling af.

W. Aukema.

Notes client startup time = super fast

The startup time of my Lotus Notes client on a Mac is fast, watch the clip I created on my Mac:

My setup: Notes 8.5 client on a Macbook Pro 17", connecting over a Ziggo cable network connection to the IBM Lotus Domino server at my office. My mailfile is not small, around 10GB.

Doing analysis at so many large customers worldwide, I sometimes hear that end users complain about the time it takes to start their Notes client. This is when I show these customers how the configuration of those clients have an impact the time it takes to startup.

Florian Vogler (Panagenda) and I decided to analyze this in more detail at a large customer. By combining data collected from workstations with MarvelClient with data that DNA collects from server logs, we expect to find evidence and best practices.

Nothing but good news about Notes this week!

On Monday, I had an insurance expert from Crawford visiting my office to inspect some water damage. As soon as the man heard about our core business, he showed a big smile on his face and proudly showed me how at Crawford all their business is done in Lotus Notes & Domino.

Asking him what he thinks about the platform, Mr. Koiter replied: "by far the best application I've ever worked with, wouldn't want anything else". He explained how he and his co-workers were able to work together on cases, share knowledge and findings and even work on the exact same document at the same time. He then gave me the example of how at a previous employer, the materials of an insurance claim were scattered all over in files, folders and shares. Consequently, team members had great difficulties finding what the were looking for, the moment they need it.

Asking about the interface, he mentioned how Crawford has a team of Notes developers who build all applications in-house. Looking at the application he had open on his tablet pc, I indeed saw a great looking interface.

Despite analyzing so many customer environments, I don't get to talk to real end users that often unfortunately. So thank you end user CN=David Koiter/O=Crawford, for sharing your experience with me.

Meet my Personal Trainer

If my medical intake shows an average BMI, my quadriceps sit-test is above average, the sit&reach test scores high and my biological age compares to that of a 31 year old (young) guy, why on earth should I start excercising?

Meet Gino, my personal trainer, he knows why: LinkYesterday had my first training session. Felt like a giraffe trying push-ups, an impala throwing baby elephants and a cheetah in the body of a hippo. Nearly managed to get back to my office when I was done, fell on the floor and asleep for an hour.

When I woke up, found myself back in reality: There is a lot of work to do!

Get Fit Stay Fit in The Hague

Fun with elephant road block in Botswana

Getting close to Chobe National Park, after driving many hours through deep sand:

Some elephants had different plans that morning though:

Pulling back about half a mile, we found a way around. At the gate, the friendly guard already suspected that elephants had blocked the road, as we were the first ones checking into the park that morning.

Their car however was not around, so I offered my assistance. 

Pictures below:

The guard promptly dismantled the entire gate, so we could use the chain...

Must admit I felt extremely good doing this...

And proud of the result...

Botswana Travel Plan Update

Hans Schoelink has a route planned.


View Larger Map

No flying backwards with Air Botswana this year. Instead we will be driving all the way, crossing the Makadikgadi Basin half-way Botswana to visit Kubu Island.

Marco de Lange managed to get us in business class ;-)

This will be our tank: Landrover Defender 130:

(Photo from www.4x4campers.co.za)

Interview about the partnership with IBM

Yesterday, Ed Brill  and I contributed to episode 162 of the famous Taking Notes Podcast by Bruce Elgort and Julian Robichaux.

Reason was that Ed Brill had publicly made the announcement (link) on his blog about the expended partnership between my company Trust Factory and IBM entered into with regards to our DNA Analysis Services.

Despite the hick-ups I caused while Ed was talking (people pinging me on Skype and my Mac started shouting that Notes needed my attention), it was a very nice and learning full experience for me.

I really think Bruce and Julian did a great job 'polishing' my input up into an excellent podcast. These guys are real pro's.

Anyway, listen for yourself (warning: the first 60 seconds are an advertisement):

Botswana here we come (again)

Tickets booked (Lufthansa) to Johannesburg, South Africa.
Toyota Land-cruiser booked, and very important:
picked up my new lens today:

This time we will start in Johannesburg and drive all the way up to Botswana, Moremi, Chobe, Kasane and then through Zimbabwe back down to South Africa.

Of course Hans & Marco are helping us out again.
(sorry for the short notice guys ;-)

The Beauty of Server Consolidation

D'66 sponsor van het RTLNieuws of gewoon toeval?

In het RTL Journaal van half acht gisteravond, viel mij op hoezeer de D'66 posters op de achtergrond in beeld kwamen, tijdens interviews met niet D'66 politici. En dit was niet de eerste keer, vorige week gebeurde precies hetzelfde. Met de verkiezingen in aantocht, deed me dit denken aan een vorm van passieve reclame...

Ik kon het daarom niet laten om Frits Wester een tweet te sturen hierover. Tot mijn (positief) stomme verbazing kreeg ik nog een reply ook:

Om mijn punt even te verduidelijken, heb ik het journaal nog even online bekeken, en daarbij een paar voorbeelden gekopieerd:

Arie Slob,  D'66?

Frits Wester, D'66?

Alexander Pechtold, D'66

Jolande Sap, D'66?

Jan Kees de Jager, D'66?

Enfin, als prominent van een van deze partijen zou ik me in de toekomst liever voor m'n eigen deur willen laten interviewen...

In Need of an SQL Factory

Having performed hundreds of DNA engagements at primarily large customers in the world, our DNA data warehouse allows us to harvest factual knowledge regarding deployment and utilization trends inside IBM Lotus Notes & Domino environments. Once the research question properly translates into an SQL select statement, we need to apply that statement across all the customer schemas from where we wish to harvest the data.

However, each DNA engagement is unique and tailored to the specific customer. So are the corresponding tables in each customer schema. Consequently, having to select all schemas to apply our query to, is a time consuming task.

I decided to automate this process. Below I'm explaining what I did.

One way to discover which schemas we can run our queries on, is to use pg_tables:

SELECT schemaname FROM pg_tables WHERE tablename = 'user_session';

Next, I needed to craft output that in itself represents an SQL statement. (Beware to escape quotes properly):

SELECT

 '

 DROP TABLE IF EXISTS '|| schemaname ||'.template_utilization\;

 CREATE TABLE '||schemaname||'.template_utilization AS

  SELECT 

    \'' || schemaname ||'\'::text AS schemaname,

    LOWER(template_used) AS inherit_from, 

    CASE 

      WHEN template_type IS NULL THEN ''Not Classified'' 

      ELSE template_type 

    END AS template_type, 

    dt.description AS database_type, 

    COUNT(*) AS num_databases, 

    COUNT(distinct db.replica_id) AS num_replicaids

  FROM

    '||SPLIT_PART(schemaname, '_', 1) || '_dwh.notes_db_detail db

    JOIN dna.database_type dt ON dt.dbtype_id = db.dbtype_id

  WHERE

    NOT db.is_removed AND db.in_scope

  GROUP BY 

    template_used, 

    template_type, 

    dt.description\;

  '::text AS SQLStatement

FROM 

  pg_tables 

WHERE tablename = 'notes_db_detail';

Executing this query outputs a row for each schema where the table exists, where each row represents the final query we can execute against the (176!) schemas we wanted. Copy the output and paste this into a new window and press F5. 

Network Sizing: Assumptions or Well Informed Decisions?

In an earlier post, I mentioned I would do some more work on the Exchange Client Bandwidth Calculator from Microsoft.

I will try and explain why I strongly disagree with these kinds of tools.

Each Domino server logs the activity performed by end users. For each user session, an entry is written in the log that records start and stop time (and duration) of the session along with the network bytes sent and received by the client from the server. There's more data logged, but this is not relevant for this article.

Because DNA collects all this data over a 7 day period, we can aggregate this session data over all activities from all end users for each DNA customer:

• kbps_rcvd = 8 x sum (bytes_received) / 1024
• kbps_sent = 8 x sum (bytes_sent) / 1024

Doing so for the most recent 100 DNA engagements, shows the average score for each customer in the graph below:

Both axes are scaled logarithmically for visualization purposes, and the overall DNA average is shown by the cross-hair. Each icon in the graph represents a customer organization, the larger the icon, the more users the organization has (seat counts range from 10 to 93,000).

Two things we can observe from these customer scores:

1. Average Client Network Bandwidth Consumption ranges from 1 to 49 kbps per user across these 100 recent DNA Customers;
2. Overall DNA Average equals 10.03 (received by users) and 2.40 kbps (sent by users);

Let's take a look at what Microsoft is doing in their calculation workbook: 

1. In the workbook (worksheet named 'Data Tables') they show peak traffic levels in kbps for each type of client
2. Microsoft assumes a normal distribution when it comes to User Demand on a typical working day

Using the default profiles in the workbook, this would result in per user averages of 6.36 kbps received for Very Heavy Users and 1.42 kbps received for Light Users. Applying this to our Real World Customers, clearly shows how this would cause significant under-capacity in 70 of our 100 recent customers:

What's causing the huge differences?

I think Microsoft makes a big mistake assuming that all users work along a normal distribution, as shown in their graph (worksheet 'Tools'):

The reality however is completely different, in several ways:

A.

Analyzing the distinct number of hours that users are active for my 100 recent customers, shows that 20% of all users are active less than two hours per day (blue line). 40% of all users are active no more than 5 hours per day.

So this means that a normal distribution -as Microsoft is assuming in their workbook- is not realistic. In fact,  the analysis of the workload for one of my customers illustrates why Microsoft is wrong:

Notice how most users are online between 7 and 17, but the network load is very much concentrated around the morning hours. This is when the remote workers (that are typically online 1-3 hours per day) consume all their new mail, especially on Monday morning.

B.

Having experience with large customers, Microsoft should realize that users can be located in different time zones. So unless you place a data center in each time zone, and have all users from that time zone connect only to that data center in their on time zone, you will see demand patterns that do not match a normal distribution. Instead you will see multiple distributions come together in one data center. 

Finally, let me show you the real world: kbps received and sent for 928,420 Lotus Notes Users  (I love Tableau Software...):

Dear Microsoft, please beware that the real distribution for network bandwidth consumption of end users lays as much as 8 decades (10^8) apart. Each customer is a unique. Even within each customer you should analyze the Real End User Demand in each office location, before calculating network requirements.

Apple TV 3, a first impression

Looking for a way to easily stream my Africa pics to my TV, I stumbled upon the Apple TV and decided to buy it for EUR 109. Package comes in a box and the form factor of the gadget is small & sexy, black matches my TV furniture ;-)

Installation equally simple, had it up & running in 5 minutes. Beware to have both an HDMI and optical audio cable available.

Currently, I have about 12,500 pics to view. In several folders. Obviously my Africa 2011 folder is the largest, with some 2,500 pics shot from a Canon 550D at 18M. 

I took me a while to find out that you don't push pics to the Apple TV, as you would expect with Airplay. Instead, you make the Apple TV pull the pics from your iTines library. Had I known sooner, I wouldn't have bought the iPhoto software at EUR 12,00 from the Apple Store :(

Here's the nice discoveries I made:

1. you make iTunes share your photo library

2. you tell the Apple TV to look at Computers and simply select the iTunes library you just shared;

3. you configure the Slideshow in the Apple TV and simply watch your pictures

4. be patient, coz the systems may need some time to sync. (this seemed to take up to a day in my case)

What's even nicer, I didn't configure music in my slide show. I simply use Spotify off my iPhone to push music into my Apple TV. Discovered that pausing the slide show with the Apple TV Remote also pauses Spotify on my iPhone.

Of course the Apple TV has more features, but I'm sure others have investigated those in far more detail than I do. If you have any questions about photo-streaming with Apple TV, feel free to reach out though.

My conclusion: I like it, and am still happy I abandoned Microsoft earlier this year.

Exchange Client Network Bandwidth Calculator

Last night one of my friends in IBM asked my opinion about the Exchange Client Network Bandwidth Calculator (it's here). I gave him my response, after I had downloaded and played with it. Here it is.

Since 10 years, my work and passion is all about analyzing network traffic for large Lotus Domino customers, I have detailed network and usage statistics for more than a million Notes users in my DNA data warehouse.  And I maintain benchmarks to compare customers against. So I think I can judge.

From a first look and play with this 2012 calculator spreadsheet, it looks pretty complete in the sense that Microsoft have incorporated a lot of variables to represent end user behavior. Now let's see what happens if we run a couple of scenario's:

Scenario A: 10,000 users with a Medium Profile

1. Leave all parameters on their initial default values in the worksheet named 'Input';
2. In the worksheet named 'Client Mix', specify all users to have a Medium Profile at each site and in such a manner that every version and type of Outlook is assigned 10,000 users;
3. Add a column to the right where you divide the resulting bandwidth by the number of users, thus calculating the kilobits per user: (=1024*Table47[@[Network Bandwidth )

Click to enlarge

Recommended network bandwidth is appr. 4 kilobits per second, per user.


Scenario B: 10,000 users with a Very Heavy Profile

1. Again leave the parameters to their initial values;
2. Change the Site User Profile to 'Very Heavy';

Click to enlarge

Recommended network bandwidth ranges from 6.5 to 11.75 kilobits per second, per user.

My observation:

Let's apply the Microsoft Recommendations to the network bandwidth requirements I reported to an organization with 9,500 users and 101 office locations world wide.

Click to enlarge

The blue bars are network demand levels observed over a 7 day period, as a consequence of measuring the real end user demand in those locations. The two red lines come from Microsoft's estimates for Medium and Very Heavy user profiles, that I applied to the user count in each of the office locations.

Of the 101 office locations, Microsoft's estimations would result in a severe shortage of network capacity for 6 locations, even when applying Microsoft's most heavy profile estimates.

Would we follow Microsoft's Calculator for Medium Profiles, more than 50% of all office locations would end up with severe shortage in their network capacity. For the remaining locations, the customer would end up with significant over capacity (56% to be exact, averaged across all sites).

Why is the Microsoft Calculator giving wrong estimates for almost every site?
Do the default user profiles not represent the behavior of users at my customer properly? Do the users in my customer send and receive more messages, and are these messages perhaps smaller or larger than what Microsoft put in their profiles? I don't think that matters. Putting more load per user profile would lead to even more over capacity at half of the sites, while still leading to shortage in others.

The real problem with this calculator is putting people into profiles. You cannot predict end user behavior, you can only observe what they are doing today. It's ok to classify your observations into profiles and categories to e.g. present pretty pictures. But it is very dangerous to use those classifications and generalize entire user populations for sizing purposes.

My Conclusion:

Predicting network requirements will almost always result in either over-capacity (waste of money) or under-capacity (end users complaining about poor performance).

With DNA, we measure the Real End User Demand over a relevant 7 day period. It is far safer to expect that end users will show the same behavior tomorrow, as we observe today, than to apply an assumption that 4 or so profiles fit all your end users.

My Recommendation:

If your sole objective is to migrate away from Domino, regardless the consequences and as quickly as possible, go ahead and use Microsoft's Network Bandwidth Calculator. You'll love it.

If you prefer to make well-informed, fact based decisions with regards to your platform strategy, based on the Real End User Demand, ask IBM to perform a detailed study on the end user demand in your organization.

Side Note:
I am not an expert in Microsoft Exchange or Outlook. Although theoretically it could be possible that network consumption of outlook users is significantly lower than a typical Notes user, I doubt that this is the case.

Cooking diner, the old fashioned 1912 style

Today I am cooking a very traditional Dutch dish called Boerenkool (Cow Cabbage or Tree Cabbage?) with Rookworst (smoked sausage), using the cookbook my grandmother Gerda Smit-Goedhart had in use when she was attending the Amsterdamsche Huishoudschool in Amsterdam around 1912. 100 years ago.

BERICHT BIJ DEN VIJFDEN DRUK
Enkele recepten werden ingelascht, op nieuwe vindingen werd de aandacht gevestigd, het gebruik van boter en eieren ingekrompen of door andere dingen vervangen, waardoor ik hoop, dat het boek aan bruikbaarheid gewonnen heeft.
C.J. WANNEE, voorheen leerares in Koken en Voedingsleer aan Die Inrichting.

The fact she describes how the use of eggs and butter was reduced to a minimum in this book-print, makes me wonder to what extend there was a shortage of these ingredients back in 1912. Was there a birth-flu and mad-cow disease happening one hundred years ago?

Reading the introduction, my eyes were caught by the following pages: How to set the table and how to serve dishes, the old-fashioned proper way.

Let me translate a couple of bullets from these pages:

1st
Extreme sanity should be in place, both on material that we will be using, personally with regards to the ones who are going to serve the dishes, as well as on the surrounding (table) where the meal is being consumed.

3rd
Rest and silence while serving, so that it does not only liberates the host from all her worries, but also positively surprises guests. One should pay particular attention to placing the glasses and plates in a silent manner, walk in a manner one cannot hear and not distract the attention of the host and hostess.

This book is about old-school rules for old fashioned dinner pleasure. A hard thing to find nowadays...

Anyway, here's the Dutch instruction for the Boerenkool dish (literally translated as Farmers Cabbage with Patatoes and Smoked Sausage):

How come this took 30-35 minutes to cook in water? Nowadays, 15-20 minutes are sufficient. Could this be the influence of today's crop manipulation?

Notes Workspace on BlackBerry

Go check this out: Teamstudio showed me a product today that brings your Notes apps to mobiles (Black Berry, iPhone, iPad, Android).

Here's a link: http://unplugged.teamstudio.com/

"Puur en Eerlijk" bij Albert Heijn

AH "Puur en Eerlijk" Scharrelvlees Slagersachterham?

Maar hier zit toch allemaal chemische troep en slechts 87% vlees in?

Biochemisch Geproduceerde Fabrieksham lijkt me een betere naam. 

Reclame Code Comissie?

(Dutch supermarket AH advertises "Pure & Honest" Ham made by old fashioned Butchers from pigs that have a lot of freedom on the farm. With only 87% meat and chemicals I believe "Biochemical Ham Derivative" would be more Pure and Honest.)

Flying People in yew York City

Just nice

Client clocking with Excel

Working together with Roy Holder of HADSL on an engagement to improve performance of a large CRM application, we needed a quick way to analyze client clock data from a Lotus Notes client.

I developed an excel macro that does the job. Here it is:

(you do need MS VbScript Regular Expressions 5.5, available in VB 6.0)


'(C) 2011 Trust Factory BV
' Wouter Aukema
Public Type parsedline
    linenr As Double
    seqThread As Integer
    seconds As Double
    seqLog As Double
    rpccall As String
    server As String
    filepath As String
    replicaid As String
    noteid As String
    ms As Double
    bytesin As Double
    bytesout As Double
    bytesall As Double
    parse_completed As Boolean
   
End Type




Sub ParseClock()
    Dim ws As Worksheet
   
    Dim ORegex As RegExp
    Dim OMatchCollection As MatchCollection
    Dim OMatch As Match
    Dim strPattern As String
   
    Dim strLine As String
    Dim origLine() As String
    Dim newLine() As String
    Dim errLine() As String
    Dim logEntry() As parsedline




    'SET ROW HEADERS IN WORKSHEET
    ActiveWorkbook.Worksheets.Add
    Set ws = ActiveSheet
    ws.Name = Format(Now, "ddmmmyyyy_hhmmss")
    ws.Cells(1, 1) = "timestamp"
    ws.Cells(1, 2) = "line_nr"
    ws.Cells(1, 3) = "seqThread"
    ws.Cells(1, 4) = "seconds"
    ws.Cells(1, 5) = "seqLog"
    ws.Cells(1, 6) = "rpccall"
    ws.Cells(1, 7) = "dbserver"
    ws.Cells(1, 8) = "dbfilepath"
    ws.Cells(1, 9) = "dbreplicaid"
    ws.Cells(1, 10) = "noteid"
    ws.Cells(1, 11) = "calloptions"
    ws.Cells(1, 12) = "milliseconds"
    ws.Cells(1, 13) = "bytesin"
    ws.Cells(1, 14) = "bytesout"
    ws.Cells(1, 15) = "bytesall"
    ws.Cells(1, 16) = "elapsedtime"
    ws.Cells(1, 17) = "elapsedbytes"
    ws.Cells(1, 18) = "elapsedkbps"
    ws.Cells(1, 19) = "design_type"
    ws.Cells(1, 20) = "design_name"
    ws.Cells(1, 21) = "design_url"
   
    Dim v As Variant
    Set v = ActiveWorkbook.CustomDocumentProperties
'    Stop
   
    txtdir = InputBox("Directory", , ActiveWorkbook.CustomDocumentProperties("directory").Value)
    If txtdir <> "" And txtdir <> ActiveWorkbook.CustomDocumentProperties("directory").Value Then ActiveWorkbook.CustomDocumentProperties("directory").Value = txtdir
    logfile = InputBox("Directory", , ActiveWorkbook.CustomDocumentProperties("logfile").Value)
    If logfile <> "" And logfile <> ActiveWorkbook.CustomDocumentProperties("logfile").Value Then ActiveWorkbook.CustomDocumentProperties("logfile").Value = logfile
   
    If logfile = "" Or txtdir = "" Then End
       
    Open txtdir + "\" + logfile For Input As #1
    'Open "C:\Program Files\lotus\notes\data\IBM_TECHNICAL_SUPPORT\xxx.log" For Input As #1
   
    Application.StatusBar = "Processing " & txtdir & "\" & logfile & " ..."
   
    'GET STARTTIMEDATE from LOGFILE
    Set ORegex = New RegExp
   
    Input #1, strLine
    ORegex.Pattern = "([0-9]{4})\_([0-9]{2})\_([0-9]{2})\@([0-9]{2})\_([0-9]{2})\_([0-9]{2})"
    Set OMatchCollection = ORegex.Execute(strLine)
        For Each OMatch In OMatchCollection
            logDate = DateSerial(Val(Left(OMatch.Value, 4)), Val(Mid(OMatch.Value, 6, 2)), Val(Mid(OMatch.Value, 9, 2)))
            logTime = TimeSerial(Val(Mid(OMatch.Value, 12, 2)), Val(Mid(OMatch.Value, 15, 2)), Val(Mid(OMatch.Value, 18, 2)))
        Next
    logStart = logDate + logTime
    ws.Cells(2, 1).NumberFormat = "dd/mm/yy hh:mm:ss"
    ws.Cells(2, 1) = logStart


    'READ ALL LINES FROM LOGFILE into origLine()
    n = 0
    While Not EOF(1)
        ReDim Preserve origLine(n)
        Line Input #1, origLine(n)
        origLine(n) = WorksheetFunction.Substitute(origLine(n), "NOTE LOCK/UNLOCK", "NOTE_LOCK_UNLOCK")
        origLine(n) = WorksheetFunction.Substitute(origLine(n), "GET LAST INDEX TIME", "GET_LAST_INDEX_TIME")
        'add more replacements when needed...
        n = n + 1
    Wend
    Close 1
   
    'NOW START WORKING LINE BY LINE, splitting multiple commands into seperate newlines()
    linenum = 2
    ReDim Preserve newLine(0)
    While linenum < UBound(origLine)
        'Check for proper lines to interpret:
        strPattern = "\([0-9]*\-[0-9]*\ \[[0-9]+\]\)\ ([A-Z0-9_]+)"
        ORegex.Pattern = strPattern
        ORegex.Global = True
        Set OMatchCollection = ORegex.Execute(origLine(linenum))
        If OMatchCollection.Count > 1 Then 'we have multiple commands on a single line...
            ReDim Preserve newLine(UBound(newLine) + 2)
            i = InStr(1, origLine(linenum), OMatchCollection.Item(1)) - 1
            newLine(UBound(newLine) - 1) = Left(origLine(linenum), i)
            newLine(UBound(newLine) - 1) = Trim(OMatchCollection.Item(0))
            'start parsing second command:
            newLine(UBound(newLine) + 0) = Trim(Mid(origLine(linenum), Len(newLine(UBound(newLine) - 1)) + 1, 200))
        ElseIf OMatchCollection.Count = 1 Then
            ReDim Preserve newLine(UBound(newLine) + 1)
            newLine(UBound(newLine)) = origLine(linenum) 'was: Trim(OMatchCollection.Item(0))
        Else
            'skip line
            'Stop
        End If
        linenum = linenum + 1
    Wend
   
    'NOW START PARSING ENTRIES
    ReDim logEntry(UBound(newLine))
    linenum = 1
    While linenum <= UBound(newLine)
        'first 4 items:
        ORegex.Pattern = "^.*\(([0-9]*)\-([0-9]*)\ \[([0-9]+)\]\)\ ([A-Z0-9_]+)"
        ORegex.Global = True
        Set OMatchCollection = ORegex.Execute(newLine(linenum))
        Set OMatch = OMatchCollection.Item(0)
        ReDim Preserve logEntry(linenum)
        logEntry(linenum).linenr = linenum
        logEntry(linenum).seqThread = OMatch.SubMatches(0)
        logEntry(linenum).seconds = OMatch.SubMatches(1)
        logEntry(linenum).seqLog = OMatch.SubMatches(2)
        logEntry(linenum).rpccall = OMatch.SubMatches(3)
       
        'replica_ids if exist...
        ORegex.Pattern = ".*REP([A-F0-9]{8})\:([A-F0-9]{8})"
        ORegex.Global = False
        Set OMatchCollection = ORegex.Execute(newLine(linenum))
        If OMatchCollection.Count > 0 Then
            Set OMatch = OMatchCollection.Item(0)
            logEntry(linenum).replicaid = OMatch.SubMatches(0) + OMatch.SubMatches(1)
        End If
       
        'note_ids if exist...
        ORegex.Pattern = ".*\-NT([A-F0-9]{8})"
        ORegex.Global = False
        Set OMatchCollection = ORegex.Execute(newLine(linenum))
        If OMatchCollection.Count > 0 Then
            Set OMatch = OMatchCollection.Item(0)
            logEntry(linenum).noteid = OMatch.SubMatches(0)
        End If
       
        'milliseconds...
        ORegex.Pattern = "\ ([0-9]+)\ ms"
        ORegex.Global = True
        Set OMatchCollection = ORegex.Execute(newLine(linenum))
        If OMatchCollection.Count > 0 Then
            Set OMatch = OMatchCollection.Item(0)
            logEntry(linenum).ms = OMatch.SubMatches(0)
        End If
       
        'bytes IO...
        ORegex.Pattern = "\[([0-9]+)\+([0-9]+)\=([0-9]+)\]"
        ORegex.Global = True
        Set OMatchCollection = ORegex.Execute(newLine(linenum))
        If OMatchCollection.Count > 0 Then
            Set OMatch = OMatchCollection.Item(0)
            If OMatch.SubMatches.Count > 1 Then
                logEntry(linenum).bytesout = OMatch.SubMatches(0)
                logEntry(linenum).bytesin = OMatch.SubMatches(1)
                logEntry(linenum).bytesall = OMatch.SubMatches(2)
            End If
        End If
       
        'Stop
        linenum = linenum + 1
    Wend
   
    'FILL WORKSHEET
    If UBound(logEntry) > 65535 Then
        MsgBox "Too many Lines!"
        End
    End If
       
        Excel.Application.ScreenUpdating = False
        Excel.Application.Calculation = xlCalculationManual
       
        ws.Cells(2, 4) = 0
    For n = 0 To UBound(logEntry)
        ws.Cells(n + 3, 2).NumberFormat = "#,##"
        ws.Cells(n + 3, 2) = logEntry(n).linenr
        ws.Cells(n + 3, 3) = logEntry(n).seqThread
        ws.Cells(n + 3, 4).NumberFormat = "#,##"
        ws.Cells(n + 3, 4) = logEntry(n).seconds
        ws.Cells(n + 3, 5).NumberFormat = "#,##"
        ws.Cells(n + 3, 5) = logEntry(n).seqLog
        ws.Cells(n + 3, 6) = logEntry(n).rpccall
        'ws.Cells(n + 3, 7) = logEntry(n).server
        'ws.Cells(n + 3, 8) = logEntry(n).filepath
        ws.Cells(n + 3, 9).NumberFormat = "@"
        ws.Cells(n + 3, 9) = logEntry(n).replicaid
        ws.Cells(n + 3, 10).NumberFormat = "@"
        ws.Cells(n + 3, 10) = logEntry(n).noteid
        ws.Cells(n + 3, 12).NumberFormat = "#,##"
        ws.Cells(n + 3, 12) = logEntry(n).ms
        ws.Cells(n + 3, 13).NumberFormat = "#,##"
        ws.Cells(n + 3, 13) = logEntry(n).bytesin
        ws.Cells(n + 3, 14).NumberFormat = "#,##"
        ws.Cells(n + 3, 14) = logEntry(n).bytesout
        ws.Cells(n + 3, 15).NumberFormat = "#,##"
        ws.Cells(n + 3, 15) = logEntry(n).bytesall
        ws.Cells(n + 3, 16).NumberFormat = "#,##.00"
        ws.Cells(n + 3, 16) = "=(RC[-15]-R2C[-15])*3600*24"
        ws.Cells(n + 3, 17).NumberFormat = "#,##"
        ws.Cells(n + 3, 17) = "=SUM(R2C[-2]:RC[-2])/1"
        ws.Cells(n + 3, 18).NumberFormat = "#,##"
        ws.Cells(n + 3, 18) = "=IF(RC[-2]>0,8*RC[-1]/RC[-2])/1024"
        ws.Cells(n + 3, 1) = "=R2C+R[-1]C[3]/24/3600"
        ws.Cells(n + 3, 1).NumberFormat = "dd/mm/yy hh:mm:ss"
        If n / 100 = Int(n / 100) Then Application.StatusBar = "Processing " & txtdir & "\" & logfile & " ... " & Str(n)
    Next n
    Excel.Application.ScreenUpdating = True
    Excel.Application.Calculation = xlCalculationAutomatic
   
    Application.StatusBar = False
End Sub


Function regreplace(rIn As String, rPattern As String, rReplace As String, Optional rIgnoreCase As Boolean, Optional rGlobal As Boolean) As String
    Dim regex As New RegExp
    Dim mc As MatchCollection
    With regex
        .Pattern = rPattern
        .IgnoreCase = rIgnoreCase
        .Global = rGlobal
    End With
    Set mc = regex.Execute(rIn)
    If mc.Count <> 0 Then
        regreplace = regex.Replace(rIn, rReplace)
    Else
        regreplace = ""
'        Debug.Print rIn
    End If
End Function