We Keep Coding

Best Performance code for huge XLS file and compare records one by one?

I read this topic more times: This SO Link about compare two XLS (Excel File), I work and try on some little examples.
I want to write a best performance C# code that read two huge XLS file and compare first row of file A with all lines of file B. if first row of A not occurred in all lines of file B, list the row of A and going to next line of A.xls and again compare with all lines of file B.
Update 1:
(I do as follows):
DataTable dt1 = GetDataTableFromExcel(this.Directory, this.FirstFile, this.FirstFileSheetName);
dtRet = getDifferentRecords(dt1, dt2);
var adapter = new OleDbDataAdapter("SELECT * FROM [" + strSheetName + "$]", connectionString);
Update 2:
My main problem occured when Xls contains 4000 records ! (Huge files)

As requested by OP, here's a VBA solution. Guessing at a few details, so OP will need to adjust to suit their specific use case
This runs for me in <2s over 4000 rows
Sub Demo()
Dim wb1 As Workbook, wb2 As Workbook
Dim ws1 As Worksheet, ws2 As Worksheet
Dim r1 As Range, r2 As Range
Dim v1 As Variant, v2 As Variant
Dim rw1 As Long, rw2 As Long
Dim cl As Long
Dim Found As Boolean
Const NUM_COLS_COMPARE = 1 'adjust as required
' Get Reference to, or open workboks
Set wb1 = Application.Workbooks("NameOfBook1.xlsx") 'if already open
Set wb2 = Application.Workbooks.Open("C:\Path\ToWorkbook2.xlsx") 'if not open
'Get reference to sheets
Set ws1 = wb1.Worksheets("NameOfSheet1")
Set ws2 = wb2.Worksheets("NameOfSheet2")
'get reference to ranges
' assuming data in Column A and Row 1fill whole range. Adjust if necassary
Set r1 = ws1.Range(ws1.Cells(1, ws1.Columns.Count).End(xlToLeft), _
ws1.Cells(ws1.Rows.Count, 1).End(xlUp))
Set r2 = ws2.Range(ws2.Cells(1, ws2.Columns.Count).End(xlToLeft), _
ws2.Cells(ws2.Rows.Count, 1).End(xlUp))
'Get Data into Array
v1 = r1.Value2
v2 = r2.Value2
For rw1 = 1 To UBound(v1, 1)
For rw2 = 1 To UBound(v2, 1)
Found = False
For cl = 1 To NUM_COLS_COMPARE
If v1(rw1, cl) = v2(rw2, cl) Then
Found = True
Exit For
End If
Next
If Found Then Exit For
Next rw2
'List Found row
If Not Found Then
Debug.Print "No Match for " & rw1, v1(rw1, 1)
End If
Next rw1
End Sub

Issue while generating number to show sequence in ssrs report

I am facing issue with SSRS report where I trying to generate numbers for showing sequence of filds. I am generating this data by using code in report. but it having two major issues.
Here I am able get numbers in my parent report but in sub report this numbering is not getting start from 1 it continues inside subreport and not considering parent report values.
My code is as given below. which will generate numbers like
1 dummy data
1.1 dummy data
1.2 dummy data
2 dummy data
2.1 dummy data
Dim currentValue As Double
Public Function GetCounter(ByVal iCounter As Double, ByVal incrementCounter
As Boolean) As Double
If (incrementCounter = true) Then
iCounter = (iCounter + currentValue)
currentValue = (currentValue + 0.1)
End If
Return iCounter
End Function
but with my sub report I want to generate numbers like
1 dummy data
1.1 dummy data
1.1.1 dummy data
1.1.1.1 dummy data
1.1.1.2 dummy data
1.1.2 dummy data
1.1.2.1 dummy data
1.1.2.2 dummy data
1.2 and so on.
I am not able get how can achive this with subReport.
Second issue is.
some fildes were I am getting correct numbers like from parrent report, having problem in PDF some extrea numbers are appearing in report and also numbers displaying in report and in PDF are diferent for some filds.
I am not able to get why this issue is comming and what solution I have to apply.
Pleas if any one knows solution to these issue please help me..

You can use the following custom code
Dim numbers = New Integer() {0, 0, 0, 0}
Public Function Seq(lev as Integer) As String
Select Case lev
Case 0
numbers(0) = numbers(0)+1
numbers(1) = 0
numbers(2) = 0
numbers(3) = 0
Return Cstr(numbers(0))
Case 1
numbers(1) = numbers(1)+1
numbers(2) = 0
numbers(3) = 0
Return Cstr(numbers(0)) & "." & Cstr(numbers(1))
Case 2
numbers(2) = numbers(2)+1
numbers(3) = 0
Return Cstr(numbers(0)) & "." & Cstr(numbers(1)) & "." & Cstr(numbers(2))
Case 3
numbers(3) = numbers(3)+1
Return Cstr(numbers(0)) & "." & Cstr(numbers(1)) & "." & Cstr(numbers(2)) &"." & Cstr(numbers(3))
End Select
End Function
The expression for group1 will be = Code.Seq(0), for group2 will be =Code.Seq(1), ... etc

multiple arraylist repeat subjectname display in sequence

Answer in c# also help me.
I tried this code for if i have duplicate string in multiple arraylist it update and display in sequence as before.
maths
english
maths
hindi
english
science
Economics
scince
i need output like this
maths_1
english_1
maths_2
hindi
science_1
Economics
scince_2
i tried this code but output is not in sequence**
Dim subjectCounts As Dictionary(Of String, Integer) = New Dictionary(Of String, Integer)
For Each subject As String In arraysub
If subjectCounts.ContainsKey(subject) Then
subjectCounts(subject) = (subjectCounts(subject) + 1)
Else
subjectCounts.Add(subject, 1)
End If
Next
Dim output As List(Of String) = New List(Of String)
For Each pair As KeyValuePair(Of String, Integer) In subjectCounts
If (pair.Value > 1) Then
Dim i As Integer = 1
Do While (i <= pair.Value)
output.Add((i.ToString + ("_" + pair.Key)))
i = (i + 1)
Loop
Else
output.Add(pair.Key)
End If
Next

I think this generates the output you want
First let us check if the subject need to have the "_#" ending
now we run throught the subject, and add the _# ending for
everyone that has more then one occurence.
The order will be the same as the input, since we run through it.
The counting will generated on the fly, so this will be correct.
Dim hasMultiple As New Dictionary(Of String, Boolean)
For Each subject As String In arraysub
If hasMultiple.ContainsKey(subject) Then
hasMultiple(subject) = True
Else
hasMultiple.Add(subject, False)
End If
Next
Dim output As New List(Of String)
Dim subCount As New Dictionary(Of String, Integer)
For Each subject As String In arraysub
If Not subCount.ContainsKey(subject) Then
subCount.Add(subject, 0)
End If
subCount(subject) += 1
If hasMultiple(subject) Then
output.Add(subject & "_" & subCount(subject))
Else
output.Add(subject)
End If
Next

Your problem is in using a dictionary. dictionaries aren't ordered so there's no guarantee of the order whenever you iterate through it. However a List(Of KeyValuePair(Of String,Integer)) will do the job you want.
Additionally you can do it using the same list(Of String). I don't use arraylist's much, never found a need that a list can't do, but I imagine the syntax should be pretty much the same. Something like this should work
Dim arraysub As List(Of String) = New List(Of String)({
"maths",
"english",
"maths",
"hindi",
"english",
"science",
"Economics",
"science"
})
For i = 0 To arraysub.Count - 1
If Not Char.IsDigit(arraysub(i).Last) Then
Dim temp As String = arraysub(i)
For j = 0 To arraysub.FindAll(Function(s) s = arraysub(i)).Count - 1
arraysub(arraysub.IndexOf(temp)) += "_" + (j + 1).ToString
Next
End If
Next
The output is:
?arraysub
Count = 8
(0): "maths_1"
(1): "english_1"
(2): "maths_2"
(3): "hindi_1"
(4): "english_2"
(5): "science_1"
(6): "Economics_1"
(7): "science_2"
Here's the same code using an arraylist:
Dim arraysub As ArrayList = New ArrayList(New String(7) {"maths", "english", "maths", "hindi", "english", "science", "Economics", "science"})
For i = 0 To arraysub.Count - 1
If Not Char.IsDigit(CStr(arraysub(i)).Last) Then
Dim temp As String = CStr(arraysub(i))
For j = 0 To Array.FindAll(arraysub.ToArray, Function(s) s Is CStr(arraysub(i))).Count - 1
arraysub(arraysub.IndexOf(temp)) = CStr(CStr(arraysub(arraysub.IndexOf(temp))) & "_" + (j + 1).ToString)
Next
End If
Next
and has this output:
?arraysub
Count = 8
(0): "maths_1" {String}
(1): "english_1" {String}
(2): "maths_2" {String}
(3): "hindi_1" {String}
(4): "english_2" {String}
(5): "science_1" {String}
(6): "Economics_1" {String}
(7): "science_2" {String}
you can see from this why many people prefer List over ArrayList.

Check this post by Charles Bretana
Create class like this,
public class MultiDimDictList<K, T>: Dictionary<K, List<T>>
{
public void Add(K key, T addObject)
{
if(!ContainsKey(key))
{
Add(key, new List<T>());
base[key].Add(addObject);
}else{
for(int i=1; i<i+1;i++){
if(!ContainsKey(key+"_"+i)){
Add(key+"_"+i+, new List<T>());
base[key+"_"+i].Add(addObject+"_"+i);
break;
}
}
}
}
}
call it like below,
myDicList.Add("YourKEY", "YourSUBJECT");
I just modified as per your requirement, but i'm not sure about this.

How to add a macro to ThisWorkbook Workbook_Open with C# Excel Interop

I developed a Windows Service using C# that processes a number of Excel files in a folder to add conditional formatting, adjust page layout and print settings and add a macro to adjust page breaks. The problem I'm having is trying to add a line of code to the ThisWorkbook object in the Workbook_Open routine to automatically run the macro when the file is opened. The code I'm using to add the macro to Module1 is as follows:
using Excel = Microsoft.Office.Interop.Excel;
using VBIDE = Microsoft.Vbe.Interop;
VBIDE.VBComponent oModule;
String sCode;
oModule = wb.VBProject.VBComponents.Add(VBIDE.vbext_ComponentType.vbext_ct_StdModule);
sCode =
#"Sub FixPageBreaks()
On Error GoTo ErrMsg
Dim wb As Workbook
Set wb = ActiveWorkbook
Dim sheet As Worksheet
Set sheet = wb.Worksheets(1)
Dim vBreaks As VPageBreaks
Set vBreaks = sheet.VPageBreaks
If vBreaks.Count > 0 Then
Dim lastCol As Integer
lastCol = ActiveSheet.Cells.Find(What:=""*"", After:=ActiveCell, LookIn:=xlFormulas, LookAt:=xlPart, SearchOrder:=xlByColumns, SearchDirection:=xlPrevious, MatchCase:=False, SearchFormat:=False).Column
Dim lCount As Integer
lCount = 1
Dim brkCol As Integer
Dim brkRng As Range
Dim iReply As VbMsgBoxResult
Do
If vBreaks(lCount).Location.Column = lastCol Then
brkCol = vBreaks(lCount).Location.Column + 1
Else
brkCol = vBreaks(lCount).Location.Column - 1
End If
Set brkRng = Range(sheet.Cells(1, brkCol), sheet.Cells(1, brkCol))
If brkCol Mod 2 = 1 And lastCol > brkCol Then
Set vBreaks(lCount).Location = brkRng
ElseIf brkCol Mod 2 = 1 Then
vBreaks(lCount).DragOff Direction:=xlToRight, RegionIndex:=1
End If
lCount = lCount + 1
Loop While lCount <= vBreaks.Count
sheet.PrintPreview
End If
Exit Sub
ErrMsg:
MsgBox Err.Description
End Sub";
oModule.CodeModule.AddFromString(sCode);
In the line
wb.VBProject.VBComponents.Add(VBIDE.vbext_ComponentType.vbext_ct_StdModule);
wb is the workbook object instantiated earlier in the code. This all works, however, I can't seem to find much documentation on the vbext_ComponentType enumeration to determine which one (if any) represent the ThisWorkbook object in the workbook and how to add code to it. I would also be happy with finding C# code that does the same thing to page breaks as the macro in the Excel document. The only reason I'm not doing it in C# like the rest of the processing is that I was unable to make it work. Any help there would be equally helpful.

var workbookMainModule = wkBk.VBProject.VBComponents.Item("ThisWorkbook");
workbookMainModule.CodeModule.AddFromString(sCode);

converting from access to MARC and back

i have a access files with details about book and i need to take the details and turn them to a marc record and vice versa. how is the best way to do it?

I released the code for my MARC implementation in C#. To do the actual coding, you will need to refer to the Library of Congress's MARC page most likely.
http://sourceforge.net/projects/marclibrary/
If you use the library, please don't hesitate to provide me feedback!
Mark

I doubt that you'll find anything that will get you there in one jump. Your best bet is most likely going to involve reading the records from Access using your favorite data access technique and then pumping it into something that speaks MARC. I haven't used it, but the MARCNet project looks promising. The MARC format isn't that difficult to implement by hand (it's all just text flat-files at heart), but a lot depends on who you are going to end up talking to and how picky they are.

FWIW, here is some code I wrote ten years ago I used process MARC and put them into an access database.
It's been a LONG time since I looked at this, but I would imagine this could be a start:
'IMPORTANT NOTE: The Marc Record's directory (this is a feature located within a Marc
' Record used to parse the record) uses option base 0 to list
' its positions for the varying fields. (See http://lcweb.loc.gov/marc)
' For example, in "Chumbawumba" the letter "C" would be in the zero position
' On the other hand, VB's Instr() function uses option base 1 to
' determine positions in a variable. ("C" is in position 1).
' Code has been adjusted to reflect this difference.
Option Explicit
Option Base 0
Const Marc21_Blank = " " 'Chr(32) Hex 20
Const Marc21_Delimiter = "" 'Chr(31) Hex 1F
Const Marc21_FieldTerminator = "" 'Chr(30) Hex 1E
Const Marc21_RecordTerminator = "" 'Chr(29) Hex 1D
Const Marc21_FillChar = "|" 'Chr(124) Hex 7C
Const Marc21_DirLength = 12 'Length of 1 direcotry Entry (option base 1)
Const Marc21_DirLengthOfFieldPos = 3 'Where the "Length of Field" number can be found in
'one directory entry (by MARC definition)
Const Marc21_DirLengthOfFieldLength = 4 'The physical length of the "Length of Field" entry
'Found in the directory
Const Marc21_DirStartingPos = 7 'Where the "Length of Field" number can be found in
'one directory entry (by MARC definition)
Const Marc21_DirStartingPosLength = 5 'The physical length of the "Length of Field" entry
'Found in the directory
Const Marc21_DirTagLength = 3 'The physical length of the "Length of Field" entry
Const Marc21_LdrLength = 24 'Length of Leader (option base 1)
Const Marc21_PositionAdjustment = 1 'Adjustment constant for Marc record (see note at top of module)
Const Marc21_FieldTerminatorPos = 1 'Represents a place for field terminator
Const Marc21_MaxControlFieldTagValue = 9 'Tags for control fields range from "001" to "009"
Const Marc21_IndicatorLength = 1 'Length of indicator
Const Marc21_MovePastIndicators = 3 'After indicators are stored in a variable field
'we should move past them to the new starting position
'This new position will be a delimeter for a variable field
Const Marc21_MovePastSubfieldCode = 2 'Used to move past the delimeter and subfield code in a varible field
'Const Marc21_FieldCount = 1 'Number of fields in Directory (option base 0)
Dim mrsLeaders As Recordset 'Recordset storing Leaders from the Marcs Imported
Dim mrsTags As Recordset 'Recordset storing the Tags
Dim mrsFields As Recordset 'Recordset storing the different variable fields in the data
'***********************************************************************************************************************
' PROCEDURE: ProcessRecord_S
'
' PURPOSE: From the text file passed, this sub will parse the Marc records
' and individually send them to the subsequent modules for further parsing
'
'PARAMETERS: sFile -- contains full path and filename of text file passed
'
'
' Date: Name: Description:
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
' 03/06/01 Ray Initial Creation
'***********************************************************************************************************************
Sub ProcessRecord_S(sFile As String)
Dim iFreeFile As Integer
Dim vMarc As Variant
Dim lRTPos As Long
Dim sMarcRecord As String
Dim dbMarc As Database
Set dbMarc = OpenDatabase("D:\Marc21\marc21.mdb")
'Made these recordsets modular because I step into two functions saving the information.
'In addition, the text file could hold thousands of records (that's a lot of loops).
Set mrsLeaders = dbMarc.OpenRecordset("Select * From NewLeaders")
Set mrsTags = dbMarc.OpenRecordset("Select * From NewTags")
Set mrsFields = dbMarc.OpenRecordset("Select * From NewFields")
lRTPos = 0 'Initialize
iFreeFile = FreeFile 'Get an available Free file number
Open sFile For Input As #iFreeFile 'Open the text file.
vMarc = Input$(LOF(iFreeFile), iFreeFile) 'Not sure what the limit is, but I have
Close #iFreeFile 'tested data as big as 10 megs.
Do Until vMarc = "" 'Going to loop till the variant is turned into an empty string
lRTPos = InStr(vMarc, Marc21_RecordTerminator) 'Since there can be more than one MarcRecord, we will use the position of the
'Record Terminator (RT) to determine where the Marc Record ends
sMarcRecord = Left(vMarc, lRTPos - 1) 'Record minus RT
If Not SaveRecord_F(sMarcRecord) Then
MsgBox "Saving Marc Record Failed"
Exit Sub
End If
vMarc = Mid(vMarc, lRTPos + 1)
Loop
End Sub
'***********************************************************************************************************************
' PROCEDURE: SaveRecord_F
'
' PURPOSE: Saves tags and indicators (if applicable) to the Tag table
'
'PARAMETERS: sMarcRecord -- Marc Record passed (this string can be up to 99,999 characters long)
'
'
' Date: Name: Description:
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
' 03/06/01 Ray Initial Creation
'***********************************************************************************************************************
Function SaveRecord_F(sMarcRecord As String)
Dim sLeader As String * 24
Dim sDirectory As String 'Whole Directory
Dim sVarFields As String 'All Control and Data Fields
Dim lDirectoryEnd As Long 'Position where Directory ends
Dim lDataEntries As Long 'Determines number of Field Entries
Dim lFieldLength As Long
Dim lStartingPosition As Long
Dim lLeaderID As Long
Dim lTagID As Long
Dim sField As String
Dim sDirectoryEntry() As String
Dim bControl As Boolean
Dim lCurDirEntry As Long
Const Marc21_FLP = 4 'Field Length position relative to directory
sLeader = Left(sMarcRecord, Marc21_LdrLength) 'Set Leader
lDirectoryEnd = InStr(Marc21_LdrLength + 1, sMarcRecord, Marc21_FieldTerminator) - Marc21_FieldTerminatorPos 'Set position for end of Directory (excluding FT)
sDirectory = Mid(sMarcRecord, Marc21_LdrLength + 1, lDirectoryEnd - Marc21_LdrLength) 'Set Directory minus field terminator
'Set Variable Fields by taking end of directory position, adding one
'for the FT position and then adding another "1" to mark the beginning of the
'vfields
sVarFields = Mid(sMarcRecord, lDirectoryEnd + Marc21_FieldTerminatorPos + 1)
lDataEntries = Len(sDirectory) 'Get Length of Directory
'Make sure Directory Length Matches with Marc21 Records
If lDataEntries Mod Marc21_DirLength <> 0 Then
MsgBox "Directory Entries are messed up"
Exit Function
End If
'Add Leader to Leader Table
mrsLeaders.AddNew
lLeaderID = mrsLeaders!LeaderID 'Need this newly assigned ID to process info below
mrsLeaders!leader = sLeader
mrsLeaders.Update
'Get Number of Directory Entries for this Marc Record
lDataEntries = lDataEntries / Marc21_DirLength
'Store Tag information and, while still looping, store Variable Field Info pertaining
'to TagID pertaining to Leader
For lCurDirEntry = 0 To lDataEntries - 1
'Starting Position of Field Entry
lStartingPosition = Val(Mid(sDirectory, lCurDirEntry * Marc21_DirLength + Marc21_DirStartingPos + Marc21_PositionAdjustment, Marc21_DirStartingPosLength))
'Field length in directory relative to current record
lFieldLength = Val(Mid(sDirectory, (lCurDirEntry * Marc21_DirLength) + Marc21_DirLengthOfFieldPos + Marc21_FieldTerminatorPos, Marc21_DirLengthOfFieldLength))
mrsTags.AddNew
mrsTags!LeaderID = lLeaderID
'According to MARC, the tag within a directory starts at position zero, so we adjust it by one for the MID function
mrsTags!Tag = Mid(sDirectory, lCurDirEntry * Marc21_DirLength + Marc21_PositionAdjustment, Marc21_DirTagLength)
'Set indicators for Data Fields (non-control fields)
If Val(mrsTags!Tag) > Marc21_MaxControlFieldTagValue Then
mrsTags!Indicator1 = Mid(sVarFields, lStartingPosition + 1, Marc21_IndicatorLength)
mrsTags!Indicator2 = Mid(sVarFields, lStartingPosition + 2, Marc21_IndicatorLength)
lStartingPosition = lStartingPosition + Marc21_MovePastIndicators
lFieldLength = lFieldLength - Marc21_MovePastIndicators 'Adjust field length accordingly
bControl = False
Else
lStartingPosition = lStartingPosition + Marc21_PositionAdjustment 'Adjusted b/c Marc21's defines positions of fields based on position zero. VB uses base 1. Imagine that.
bControl = True
End If
lTagID = mrsTags!TagID
'mrsTags!FieldDesc = Mid(sVarFields, lStartingPosition + 1, lFieldLength - 1) 'Directory Entry - Field Termintor (FT)
mrsTags.Update
'Only Get field pertaining to Direcotry Entry Just Saved
sField = Mid(sVarFields, lStartingPosition, lFieldLength)
SaveFields_F sField, lTagID, bControl 'Adding and Subtracting two is accounting for indicators
Next lCurDirEntry
SaveRecord_F = True
End Function
'***********************************************************************************************************************
' PROCEDURE: SaveFields_F
'
' PURPOSE: Saves the variable data and control fields to its respective table
'
'PARAMETERS: sField -- The data which must be filtered
' lTagID -- The unique identifier related to the records about to be stored
' bControl -- Optional. Let's sub know if data is a control field or not
'
'
' Date: Name: Description:
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
' 03/06/01 Ray Initial Creation
'***********************************************************************************************************************
Function SaveFields_F(sField As String, lTagID As Long, Optional bControl As Boolean)
Dim lDelimeterPos As Long
If Not bControl Then 'Not a control Field
If Left(sField, 1) = Marc21_Delimiter Then
sField = Mid(sField, 2) 'Move past delimeter
Else
'All non-control fields should start with delimeters
MsgBox "This entry didn't start out with a delimeter", vbOKOnly, "Uh Oh"
Exit Function
End If
lDelimeterPos = InStr(1, sField, Marc21_Delimiter)
Do Until lDelimeterPos = 0
'Store new subfield code and field description
mrsFields.AddNew
mrsFields!TagID = lTagID
mrsFields!SubFieldCode = Left(sField, 1)
mrsFields!FieldDesc = Mid(sField, Marc21_MovePastSubfieldCode, lDelimeterPos - Marc21_MovePastSubfieldCode) 'Start at position two b/c of subfield code. Subtract by 2 b/c of where started
mrsFields.Update
sField = Mid(sField, lDelimeterPos + 1)
'Get new delimeter position
lDelimeterPos = InStr(1, sField, Marc21_Delimiter)
Loop
If sField <> "" Then
mrsFields.AddNew
mrsFields!TagID = lTagID
mrsFields!SubFieldCode = Left(sField, 1)
mrsFields!FieldDesc = Mid(sField, Marc21_MovePastSubfieldCode)
mrsFields.Update
End If
Else
If sField <> "" Then
If Right(sField, 1) = Marc21_FieldTerminator Then
sField = Mid(sField, 1, Len(sField) - Marc21_FieldTerminatorPos)
End If
mrsFields.AddNew
mrsFields!TagID = lTagID
mrsFields!FieldDesc = sField
mrsFields.Update
End If
End If
End Function
The access database I created (and used) can be found here.