I have a function to create System.Data.Datatables from a simple input of TableName and ColumnName-Array. After filling the tables I want to add them to a dataset and export this one into an excel document.
The code below does that, however in the export to excel there is one bit that I find a bit unelegant. Every table will have to be exported into a csv and then reimported to excel.
Is there a better cleaner way to use DataTables directly in Excel?
Function MakeTable ($TableName, $ColumnArray)
{
$btab = New-Object System.Data.DataTable("$TableName")
foreach($Col in $ColumnArray)
{
$MCol = New-Object System.Data.DataColumn $Col;
$btab.Columns.Add($MCol)
}
return , $btab
}
function DataSetToExcel ($Ds, $workdirectory)
{
$excel = New-Object -ComObject excel.application
$workbook = $excel.Workbooks.Add(1)
$i = 0
for($DsIndex=0;$DsIndex -lt $ds.Tables.Count;$DsIndex++)
{
$Table = $ds.tables[$Dsindex]
if($Dsindex -ne 0)
{
$workbook.worksheets.Add() | Out-Null #Erstellt neues Arbeitsblatt
}
$Table | Export-Csv "$workdirectory\input.csv" -Encoding UTF8 -NoTypeInformation -Force -Delimiter ";"
$inputCSV = "C:\Temp\Test\input.csv"
$worksheet = $workbook.worksheets.Item(1)
$worksheet.Name = $Table.TableName
$TxtConnector = ("TEXT;" + $inputCSV)
$Connector = $worksheet.QueryTables.add($TxtConnector,$worksheet.Range("A1"))
$query = $worksheet.QueryTables.item($Connector.name)
$query.TextFileOtherDelimiter = $Excel.Application.International(5)
$query.TextFileParseType = 1
$query.AdjustColumnWidth = 1
$query.Refresh() | Out-Null
$query.Delete()
}
$outputXLSX = "$workdirectory\output.xlsx"
$Workbook.SaveAs($outputXLSX,51)
$excel.Quit()
}
function MakeTestTable ($TableName)
{
$TestTable = MakeTable $TableName #("A","B")
for($i=0;$i -lt 10; $i++)
{
$aRow = $TestTable.NewRow()
$aRow["A"] = (10-$i).ToString()
$aRow["B"] = $i.ToString()
$TestTable.Rows.Add($aRow)
}
return , $TestTable
}
$db = New-Object System.Data.DataSet
for($cx=0;$cx -lt 10;$cx++)
{
$tab1 = MakeTestTable "$cx"
$db.Tables.Add($tab1)
}
DataSetToExcel $db "C:\Temp\Test"
Check out my PowerShell Excel Module on Github. You can also grab it from the PowerShell Gallery.
It also works directly with CSV format using Import-Csv or ConvertFrom-Csv (basically any PowerShell object array).
function New-Person {
param($First,$Last)
$row=$dataTable.NewRow()
$row["First"]=$First
$row["Last"]=$Last
$dataTable.Rows.Add($row)
}
$dataTable = New-Object System.Data.DataTable("Test")
$dataTable.Columns.Add((New-Object System.Data.DataColumn "First"))
$dataTable.Columns.Add((New-Object System.Data.DataColumn "Last"))
New-Person John Doe
New-Person Tom Doe
New-Person Jane Doe
New-Person Mary Doe
$dataTable |
Select First, Last |
Export-Excel c:\temp\people.xlsx -AutoSize -Show
Since you tagged this with C#, If you have excel on the computer this will be run on, you can use the interop library. I used it for a project I did, using code I found in this SO question.
Related
Developing an C# project for SQL Training and giving different exercises based on training on each topic. One of the exercise is to write a query using Sub-Query. which needs to be evaluated whether the user has used/implemented Sub query in the Query Statment.
Q: Write a sql query to show the SalesOrderID,LineTotal,average LineTotal from the Sales.SalesOrderDetail table using Sub query
Select SalesOrderID,LineTotal [LineTotal],
(Select AVG(LineTotal) from Sales.SalesOrderDetail) as [AverageLineTotal]
from Sales.SalesOrderDetail
[AverageLineTotal] is an sub query.
Can we identify it by any means?? like execution plan Or SP to identify it has an sub query in the statement
Is there any way to identify it through execution Plans??
If this is a c# project you can parse the query with regex to find if the query contains (select {any other text}).
public static void Main()
{
var sql = #"Select SalesOrderID,LineTotal [LineTotal],(Select AVG(LineTotal) from Sales.SalesOrderDetail) as [AverageLineTotal] from Sales.SalesOrderDetail";
Console.WriteLine(DoesSqlContainSubquery(sql));
}
public bool DoesSqlContainSubquery(string sql)
{
var regexTest = new Regex(#"\( *Select .*\)", RegexOptions.IgnoreCase);
var containsSubquery = regexTest.IsMatch(sql);
return containsSubquery;
}
Parsing ad-hoc scripts is inherently complex due to the plethora T-SQL constructs and options. That being said, a robust method for targeted use cases is parsing scripts with the Microsoft.SqlServer.TransactSql.ScriptDom.
Below is an example PowerShell script that uses the script DOM assembly from the official Microsoft Dacfx NuGet package, downloading and extracting it if needed.
# Add TSqlScript DOM assembly reference, downloading and extracting to the specified location if needed
$scriptDomAssemblyPath = "C:\Temp\Microsoft.SqlServer.TransactSql.ScriptDom.dll"
$scriptDomNuGetUrl = "https://www.nuget.org/api/v2/package/Microsoft.SqlServer.DacFx.x64/150.4384.2"
if(![System.IO.File]::Exists($scriptDomAssemblyPath)) {
$response = Invoke-WebRequest -Uri $scriptDomNuGetUrl
if ($response.StatusCode -ne 200) {
throw "Unable to download Microsoft.SqlServer.TransactSql.ScriptDom NuGet package: $response.StatusCode : $response.StatusDescription"
}
$tempZipFilePath = "$([System.IO.Path]::GetTempPath())/$([System.IO.Path]::GetRandomFileName()).zip"
[System.IO.File]::WriteAllBytes($tempZipFilePath, $response.Content)
$response.BaseResponse.Dispose()
$tempUnzipFolderPath = "$([System.IO.Path]::GetTempPath())/$([System.IO.Path]::GetRandomFileName())"
Expand-Archive -Path $tempZipFilePath -DestinationPath $tempUnzipFolderPath
$tempZipFilePath | Remove-Item
Move-Item "$tempUnzipFolderPath\lib\net46\Microsoft.SqlServer.TransactSql.ScriptDom.dll" "$scriptDomAssemblyPath"
$tempUnzipFolderPath | Remove-Item -Recurse
}
Add-Type -Path $scriptDomAssemblyPath
# script to be parsed
$scriptText = #"
Select SalesOrderID,LineTotal [LineTotal],
(Select AVG(LineTotal) from Sales.SalesOrderDetail) as [AverageLineTotal]
from Sales.SalesOrderDetail
"#
#parse script
$parser = New-Object Microsoft.SqlServer.TransactSql.ScriptDom.TSql150Parser($true)
$parseErrors = New-Object System.Collections.Generic.List[Microsoft.SqlServer.TransactSql.ScriptDom.ParseError]
$scriptReader = New-Object System.IO.StringReader($scriptText)
$script = $parser.Parse($scriptReader, [ref]$parseErrors)
if($parseErrors.Count -gt 0) {
throw "$($parseErrors.Count) parsing errors"
}
# sanity check for expected SELECT query
if(($script.Batches.Count -ne 1) -or ($script.Batches[0].Statements.Count -ne 1) -or ($script.Batches[0].Statements[0].QueryExpression -eq $null)) {
throw "script with single SELECT statement expected"
}
# find scalar subquery expression in select list
$subQueryFound = $false
foreach($selectElement in $script.Batches[0].Statements[0].QueryExpression.SelectElements) {
if($selectElement.Expression.ToString() -eq "Microsoft.SqlServer.TransactSql.ScriptDom.ScalarSubquery") {
$subQueryFound = $true
break
}
}
# show if subquery was used
if($subQueryFound) {
Write-Host "A subquery is used"
}
else {
Write-Host "A subquery is not used"
}
How to find duplicate values in Excel and export rows to another sheet using power shell?
I had an Excel sheet with multiple rows and column lets say from "A" to "k". I need to find duplicate rows only if values in all the columns in a row are unique. And the script should ignore D,E,F columns even though those column's values are same.
The script should also copy all those duplicate rows and should paste in a new excel file.it should also copy header row and source of duplicate rows and Also attaching a sample image of input file(the output file should also be same as input in this input case because it should also copy source duplicate rows). I had tried a code but it is throwing an error..please look into that and give me a solution for the code.
code:
# The Text OleDB driver is only available in PowerShell x86. Start x86
shell if using x64.
# This has to be the first check this script performs.
if ($env:Processor_Architecture -ne "x86") {
Write-Warning "Switching to x86 shell"
&"$env:windir\syswow64\windowspowershell\v1.0\powershell.exe"
"$PSCommandPath $args"; return
}
# Change to your CSV file name, must end in .csv or .tsv
$csvfile = "C:\files\A01modcsv.csv"
# Does the first row contain column names?
$firstRowColumns = $True
# What's the delimiter? Use `t for tabbed.
$csvdelimter = "`t"
$firstRowColumns = $true
$checkColumns = "A"
$datasource = Split-Path $csvfile
$tablename = (Split-Path $csvfile -leaf).Replace(".","#")
switch ($firstRowColumns) {
$true { $firstRowColumns = "Yes" }
$false { $firstRowColumns = "No" }
}
$elapsed = [System.Diagnostics.Stopwatch]::StartNew()
[void][Reflection.Assembly]::LoadWithPartialName("System.Data")
# Setup OleDB using Microsoft Text Driver.
$connstring = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=$datasource;Extended Properties='text;HDR=$firstRowColumns;FMT=Delimited($csvdelimter)';"
$conn = New-Object System.Data.OleDb.OleDbconnection
$conn.ConnectionString = $connstring
$conn.Open()
$cmd = New-Object System.Data.OleDB.OleDBCommand
$cmd.Connection = $conn
# Perform select on CSV file, then add results to a datatable using ExecuteReader
$sql = "SELECT $checkColumns, COUNT(*) as DupeCount FROM [$tablename] GROUP BY $checkColumns HAVING COUNT(*) > 1"
$cmd.CommandText = $sql
$dt = New-Object System.Data.DataTable
$dt.BeginLoadData()
$dt.Load($cmd.ExecuteReader([System.Data.CommandBehaviour]::CloseConnection))
$dt.EndLoadData()
$totaltime = [math]::Round($elapsed.Elapsed.TotalSeconds,2)
# Get Total Row Count
$cmd.CommandText = "SELECT COUNT(*) as TotalRows FROM [$tablename]"
$totalrows = $cmd.ExecuteScalar()
$conn.Close()[enter image description here][1]
# Output some stats
$dupecount = $dt.Rows.Count
Write-Host "Total Elapsed Time: $totaltime seconds. $dupecount duplicates found out of $totalrows total rows. You can access these dupes using `$dt." -ForegroundColor Green
I am getting an error in the menctioned code at "$dt.Load($cmd.ExecuteReader([System.Data.CommandBehaviour]::CloseConnection))" command....can any help me in solving this error.Thank you.
Trying to execute a PowerShell command in asp.net c# but it returns no results, where am I going wrong?
var shell = PowerShell.Create();
var script = $#"$Groups = Get-ADGroup {groupname}; $members = ForEach ($Group in $Groups) {{Get-AdGroupMember -Identity $Group -Recursive}} ; $members | Get-AdUser -Properties Department | Select-Object Name, Department | Sort Department, Name";
shell.Commands.AddScript(script);
var results = shell.Invoke();
foreach (var psObject in results)
{
dt.Rows.Add(new object[] { psObject.Members["Name"].Value, psObject.Members["Department"].Value });
}
Gridview2.DataSource = dt;
Gridview2.DataBind();
It executes perfectly in PowerShell. Separating the command over lines as per suggested answer does not work.
Edit - I took another look at my powershell and realised that it was unecessarily complicated. I changed it to the below and it now works. Still does not explain why something that returns results in powershell does not work when ran from asp.net.
Get-AdGroupMember -Identity {groupname} -Recursive | Get-AdUser -Properties Department | Select-Object Name, Department | Sort Department, Name
I have this query:
string query = "SELECT afspraak_locatie FROM Afspraak WHERE date(datum) = '" + datum +"'";
The final query will look like this:
SELECT afspraak_locatie FROM Afspraak WHERE date(datum) = '2016-06-16'
When i execute the query in my PHPMYADMIN it returns the row. But when i do it in C# it says my MySqldatareader is empty
Here is the code i use for that:
MySqlCommand cmd1 = new MySqlCommand(query1, connection);
cmd1.CommandType = CommandType.Text;
using (MySqlDataReader reader1 = cmd1.ExecuteReader())
{
while (reader1.Read())
{
result1.Add(reader1.GetString(0));
}
reader1.Close();
}
cmd1.Cancel();
When this gets executed it will give a System.NullreferenceException on the while(reader1.read) part. Any solutions?
Schema and data loaded:
create table Afspraak
(
id int auto_increment primary key,
afspraak_locatie varchar(100) not null, -- just an example (we don't know your datatype)
datum datetime not null -- you said it was a datetime in a comment under your question
);
insert Afspraak (afspraak_locatie,datum) values
('Rome','2016-06-14 13:55:55'),
('London','2016-06-15 15:12:12'),
('Cairo','2016-06-16 07:00:33'),
('Boston','2016-06-17 01:30:00');
select * from afspraak;
+----+------------------+---------------------+
| id | afspraak_locatie | datum |
+----+------------------+---------------------+
| 1 | Rome | 2016-06-14 13:55:55 |
| 2 | London | 2016-06-15 15:12:12 |
| 3 | Cairo | 2016-06-16 07:00:33 |
| 4 | Boston | 2016-06-17 01:30:00 |
+----+------------------+---------------------+
GUI Layer:
private void button1_Click(object sender, EventArgs e)
{
myDB.FindThatRow("2016-06-16"); // get data
}
DB Layer:
public void FindThatRow(string theDate)
{ // or all those rows
//
using (MySqlConnection lconn = new MySqlConnection(connString))
{
lconn.Open();
using (MySqlCommand cmd = new MySqlCommand())
{ //
cmd.Connection = lconn;
cmd.CommandText = #"select id,afspraak_locatie FROM Afspraak WHERE date(datum) = #pTheDate";
cmd.Prepare();
cmd.Parameters.AddWithValue("#pTheDate", theDate);
using (MySqlDataReader rs = cmd.ExecuteReader())
{ //
while (rs.Read())
{
int qId = (int)rs.GetInt32("id");
string sViewIt = rs.GetString("afspraak_locatie");
}
}
}
}
}
It found the data:
Use the using blocks as recommended by everyone. Bind your parameters.
The reasons why one should steer toward data bindings, versus string concatenation as seen in your attempt, include losing the functionality of what binding offers as seen in Configuring Parameters and Parameter Data Types and other links near or off that topic. And, it turns querying into the mess seen in PHP with concatenation which steered their modern usage toward parameter data bindings too.
Imagine how difficult and debug-intensive the following query would be without bindings:
Sql Injection Attacks:
Parameter binding protects you from such attacks, unlike your method of concat. See the following question including this answer for stored procedure usage.
This is the powershell script I want to run in c# :
ForEach ($Mailbox in Get-Mailbox) {Get-ActiveSyncDeviceStatistics -Mailbox
$Mailbox.Identity –ErrorAction SilentlyContinue |
SelectDeviceFriendlyName,Devicetype,DeviceUserAgent | ForEach-Object { $_ | Add-Member –
MemberType NoteProperty -Name "MailboxIdentity" -value $Mailbox}}
I could complete the c# coding upto first foreach loop in powershell script, but I do not know how implement the 2nd foreach that uses $mailbox variable along with the coding given below.
PowerShell powershell = PowerShell.Create();
PSCommand command = new PSCommand();
command.AddCommand("Get-Mailbox");
command.AddCommand("where-object");
command.AddParameter("Filterscript", ScriptBlock.Create("!$_.name.startswith \"DiscoverySearchMailbox\")"));
powershell.Commands = command;
powershell.Runspace = CreateRunSpace.GetRunSpace();
var result = powershell.Invoke();
PSCommand command1 = new PSCommand();
command1.AddCommand("write-output");
command1.AddParameter("InputObject", result);
command1.AddCommand("Foreach-Object");
command1.AddParameter("Process", ScriptBlock.Create("Get-ActiveSyncDeviceStatistics -Mailbox $_.Identity"));
powershell.Commands = command1;
powershell.Runspace =
CreateRunSpace.GetRunSpace();
var result1 = powershell.Invoke();
I'm stuck at 2nd foreach loop to add-member while adding the output of get-mailbox.
Appreciate your help and suggestions in advance. Thank you.