We Keep Coding

splitting strings in oracle query

I want to split my strings in Oracle based on length with space as a delimiter.
For example,
`MY_STRING="Before continuing, turn off the top title display without changing its definition:"`
My output should be
`STRING1="Before continuing, turn off the"`
`STRING2="top title display without changing"`
`STRING3="its definition:"`
The strings should be a maximum of 35 characters in length. The words after position 105 can be ignored.

It colud be done with a stored function :
create or replace FUNCTION get_part(p_value IN VARCHAR2, part in number)
RETURN VARCHAR2
IS temp VARCHAR2(1000);
BEGIN
temp := p_value;
FOR i IN 1 .. (part-1) LOOP
if (Length(temp) <35) then
return '';
ELSE
FOR j in REVERSE 1 .. 35 LOOP
if SUBSTR(temp,j,1) = ' ' then
temp := SUBSTR(temp,j+1);
EXIT;
end if;
END LOOP;
temp := SUBSTR(temp,36);
end if;
END LOOP;
if (Length(temp) <=35) then
return temp;
else
FOR j in reverse 1 .. 35 LOOP
if SUBSTR(temp,j,1) = ' ' then
return SUBSTR(temp,1,j-1);
end if;
END LOOP;
return SUBSTR(temp,1,35);
end if;
END;
usage:
select
get_part(string_value,1),
get_part(string_value,2),
get_part(string_value,3) from ( select 'Before continuing, turn off the top title display without changing its definition:' string_value from dual)
It surely will fail if there are more than 35 chars without space, i'll leave that to you
EDIT: now it should split hard after 35 chars if there are no spaces

You have tagged C# so i chose the language to answer you. I hope it helps. It perfectly splits the text.
Regarding the rules, in your case it splits the text in 3 parts.
This is the Outcome :
STR1 :"Before continuing, turn off the"
STR2 :" top title display without changing"
STR3 :" its definition:"
static void Main(string[] args)
{
const string txt = "Before continuing, turn off the top title display without changing its definition:";
var txtArr = txt.ToCharArray();
var counter = 0;
var stringList = new List<string>();
var str = string.Empty;
for (var i = 0; i < txt.Count(); i++)
{
counter++;
if (counter == 35)
{
while (txtArr[i].ToString() != " ")
{
i--;
str = str.Remove(i);
}
stringList.Add(str);
str = string.Empty;
counter = 0;
}
str = str + txtArr[i];
}
stringList.Add(str);
}
This is how I implemented the algorithm in ORACLE (PL/SQL). Ignore the error and look at the output. It returns 3 lines and works properly. Now write some extra code and modify it as you want. The error does not seem important and I have no idea what the reason is.
declare
--
txt nvarchar2(1000):='Before continuing, turn off the top title display without changing its definition:';
charc nvarchar2(1):='';
TYPE txtArrTyp IS VARRAY(1000) OF NVARCHAR2(1);
txtArr txtArrTyp :=txtArrTyp();
--
str nvarchar2(35):='';
cntr number:=0;
j number:=0;
lent number:=0;
begin
--
lent:=LENGTHB(txt);
--
for i In 1 ..lent
loop
if(txt is null )then
dbms_output.put_line('SHIT');
end if;
charc := SUBSTR(txt,i,1);
txtArr.extend;
txtArr(i):=charc;
end loop;
--
While(j>=1 or j<=lent)
loop
j:=j+1;
cntr :=cntr+1;
if(cntr = 35) then
while(txtArr(j)<>' ')
loop
j:=j-1;
end loop;
str:=substr(str,0,j);
dbms_output.put_line(str);
str:=null;
cntr:=0;
end if;
str := str || txtArr(j);
end loop;
dbms_output.put_line(str);
end;

SQL Server: regex search capture group count

I'm currently working on a search method in C# for a SQL Server database.
The regex:
/(a)|(b)|(c)|(d)/g
a, b, c & d are the search keywords.
The string that I apply the regex to:
a fdh eidb
Consists of random words(represented as letters) and some of the keywords from above.
Expected output:
3 keywords matches.
But how does a SQL query for SQL Server look like that returns a table with a Matches column with the keyword match count for each row?
I know how to do this in C# but I want to do it in the search query itself so I can sort the output.
Thanks for any help towards the right direction :)

I don't think there is a way to do regular expressions in SQL Server queries - other than adding some managed code which adds that functionality.
Here is an example of how to do that - SQL Server Regular expressions in T-SQL

It seems that REGEX wasn't really the solution.
Instead I wrote multiple SQL functions that do the job:
CREATE FUNCTION [dbo].[KeywordMatches]
(
#String nvarchar(1000),
#Keywords nvarchar(1000),
#Seperator text
)
RETURNS INT
AS
BEGIN
DECLARE #Count int = 0;
DECLARE #Keyword varchar(1000);
DECLARE KeywordsCursor CURSOR FOR
SELECT *
FROM [dbo].StringSplit(#Keywords, #Seperator)
OPEN KeywordsCursor
FETCH NEXT FROM KeywordsCursor INTO #Keyword
WHILE ##FETCH_STATUS = 0
BEGIN
IF #String LIKE '%' + #Keyword + '%'
SET #Count += 1
FETCH NEXT FROM KeywordsCursor INTO #Keyword
END
CLOSE KeywordsCursor
DEALLOCATE KeywordsCursor
RETURN #Count
END
And (fallback for server 2016 split_string):
CREATE FUNCTION [dbo].[StringSplit]
(
#SeperatedWords nvarchar(1000),
#Seperator char
)
RETURNS #Words TABLE
(
Word nvarchar(1000)
)
AS
BEGIN
DECLARE #Position int = -1
SET #SeperatedWords += #Seperator
WHILE (#Position > 0 OR #Position = -1)
BEGIN
SET #SeperatedWords = SUBSTRING(#SeperatedWords, #Position + 1, LEN(#SeperatedWords) - #Position + 1)
SET #Position = CHARINDEX(#Seperator, #SeperatedWords)
/* Only add words that have a length bigger then 0 */
IF #Position > 1
/* Add the word to the table */
INSERT INTO #Words(Word) VALUES(LEFT(#SeperatedWords, #Position - 1))
END
RETURN
END
Usage:
SELECT Id, Title, [dbo].KeywordMatches(Title, 'blue red green', ' ') AS Matches
FROM Questions
ORDER BY Matches DESC, Date DESC
Above query orders by the amount of keywords found in the title and date.
I also read about full text search which is probably faster then this solution.

How can I deal with parsing bad csv data?

I know that the data should be correct. I have no control over the data and my boss is just going to tell me that I need to figure out a way to deal with someone else's mistake. So please don't tell me it's not my problem that the data is bad, because it is.
Anywho, this is what I'm looking at:
"Words","email#email.com","","4253","57574","FirstName","","LastName, MD","","","576JFJD","","1971","","Words","Address","SUITE "A"","City","State","Zip","Phone","",""
Data has been scrubbed for confidentiality reasons.
So as you see, the data contains quotation marks and there are commas inside some of these quoted fields. So I cannot remove them. But the "Suite A""" is throwing off the parser. There are too many quotation marks. >.<
I'm using the TextFieldParser in the Microsoft.VisualBasic.FileIO namespace with these settings:
parser.HasFieldsEnclosedInQuotes = true;
parser.SetDelimiters(",");
parser.TextFieldType = FieldType.Delimited;
The error is
MalformedLineException: Line 9871 cannot be parsed using the current
delimiters.
I would like to scrub the data somehow to account for this but I'm not sure how to do it. Or maybe there's a way to just skip this line? Although I suspect my higher ups will not approve of me just skipping data that we might need.

If you are only trying to get rid of the stray " marks in your csv, you can use the following regex to find them and replace them with '
String sourcestring = "source string to match with pattern";
String matchpattern = #"(?<!^|,)""(?!(,|$))";
String replacementpattern = #"$1'";
Console.WriteLine(Regex.Replace(sourcestring,matchpattern,replacementpattern,RegexOptions.Multiline));
Explanation:
#"(?<!^|,)""(?!(,|$))"; will find will find any " that is not preceded by the beginning of the string, or a , and that is not followed by the end of the string or a ,

I am not familiar with TextFieldParser. However with CsvHelper, you can add a custom handler for invalid data:
var config = new CsvConfiguration();
config.IgnoreReadingExceptions = true;
config.ReadingExceptionCallback += (e, row) =>
{
// you can add some custom patching here if possible
// or, save the line numbers and add/edit them manually later.
};
using(var file = File.OpenRead(".csv"))
using(var reader = new CsvReader(reader, config))
{
reader.GetRecords<YourDtoClass>();
}

My only addition to what everyone is saying (because we've all been there) is to try to attempt to rectify each new issue you encounter with code. There are some decent REGEX strings out there https://www.google.com/?ion=1&espv=2#q=c-sharp+regex+csv+clean or you could manually fix things using String.Replace (String.Replace("\"\"\"","").Replace("\"\","").Replace("\",,","\",") or such). Eventually, as you detect and find ways of correcting more and more mistakes, your manual recovery rate will be minimized substantially (most of your bad data will likely come from similar mistakes). Cheers!
PS - Idea-ish (it's been a while - the logic may neeed some tweaking as I'm writing from memory), but you'll get the gist:
public string[] parseCSVWithQuotes(string csvLine,int expectedNumberOfDataPoints)
{
string ret = "";
string thisChar = "";
string lastChar = "";
bool needleDown = true;
for(int i = 0; i < csvLine.Length; i++)
{
thisChar = csvLine.Substring(i, 1);
if (thisChar == "'"&&lastChar!="'")
needleDown = needleDown == true ? false : true;//when needleDown = true, characters are treated literally
if (thisChar == ","&&lastChar!=",") {
if (needleDown)
{
ret += "|";//convert literal comma to pipe so it doesn't cause another break on split
}else
{
ret += ",";//break on split is intended because the comma is outside the single quote
}
}
if (!needleDown && (thisChar == "\"" || thisChar == "*")) {//repeat for any undesired character or use RegEx
//do not add -- this eliminates any undesired characters outside single quotes
}
else
{
if ((lastChar == "'" || lastChar == "\"" || lastChar == ",") && thisChar == lastChar)
{
//do not add - this eliminates double characters
}else
{
ret += thisChar;
lastChar = thisChar;
//this character is not an undesired character, is no a double, is valid.
}
}
}
//we've cleaned as best we can
string[] parts = ret.Split(',');
if(parts.Length==expectedNumberOfDataPoints){
for(int i = 0; i < parts.Length; i++)
{
//go back and replace the temporary pipe with the literal comma AFTER split
parts[i] = parts[i].Replace("|", ",");
}
return parts;
}else{
//save ret to bad CSV log
return null;
}
}

I've had to do this before,
The first step is to parse the data using string.split(',')
The next step is to combine the segments that belong together.
What I essentially did was
make a new list representing the combined strings
if a string begins with a quote, push it onto your new list
if it does not begin with a quote, append it to the last string in your list
Bonus: throw exceptions when a string ends with a quote but the next one does not begin with a quote
Depending on what the rules are regarding what can actually appear in your data, you might have to change your code to account for that.

At the core of CSV's file format, each line is a row, each cell in that row is separated by a comma. In your case, your format also contains the (very unfortunate) stipulation that commas inside a pair of quotation marks do not count as separators and are instead part of the data. I say very unfortunate because a misplaced quotation mark affects the entire rest of the line, and since quotation marks in standard ASCII do not distinguish between open and closed, there really is nothing you can do to recover from this without knowing the original intent.
That is when you log a message in a way that the person who does know the original intent (the person that provided the data) can look at the file and correct the error:
if (parse_line(line, &data)) {
// save the data
} else {
// log the error
fprintf(&stderr, "Bad line: %s", line);
}
And since your quotation marks aren't escaping newlines, you can keep on going with the next line after running into this error.
ADDENDUM: And if your company has a choice (i.e. your data is being serialized by a company tool) don't use CSV. Use something like XML or JSON with a much more clearly defined parsing mechanism.

I had to do this once aswell. My approach was to go through a line and keep track on what I was reading.
Basicly, I coded my own scanner chopping off tokens from the input line which gave me full control over my faulty .csv data.
This is what I did:
For each character on a line of input.
1. when outside of a string meeting a comma => all of the previous string (which can be empty) is a valid token.
2. when outside of a sting meeting anything but a comma or a quote => now you have a real problem, unquoted tekst => handle as you see fit.
3. when outside of a string meeing a quote => found a start of string.
4. when inside of a string meeting a comma => accept the comma as part of the string.
5. when inside of the string meeting a qoute => trouble starts here, mark this point.
6. continue and when meeting a comma (skipping white space if desired) close the string, 'unread' the comma and continue. (than will bring you to point 1.)
7. or continue and when meeting a quote -> obviously, what was read must be part of the string, add it to the string, 'unread' the quote and continue. (that will you bring to point 5)
8. or continue and find an whitespace, then End Of Line ('\n') -> the last qoute must be the closing quote. accept the string as a value.
9. or continue and fine non-whitespace, then End Of Line. -> now you have a real problem, you have the start of a string but it is not closed -> handle the error as you see fit.
If the number of fields in your .csv file is fixed you can count the comma's you recognise as field seperators and when you see a End Of Line you know you have another problem or not.
With the stream of strings received from the input line you can build a 'clean' .csv line and this way build a buffer of accepted and cleaned input that you can use in your already existing code.

What is wrong with this regex, not working correctly?

My query is working fine inside my oracle, but when I add it inside my oracle command getting error at [A-Z] and \1\3 saying unrecognized character. I think I need to use # some where to make it correct but I don't know where?
*My Query to see definition of trigger without user name: also deleting line that start with ALTER TRIGGER.. *
OracleCommand Command = new OracleCommand(#"SELECT regexp_replace(dbms_metadata.get_ddl('TRIGGER','" + triggernames + "'),'(CREATE OR REPLACE TRIGGER )("[A-Z]+"\.)(.+)(ALTER TRIGGER .+)','\1\3', 1, 0, 'n')FROM dual", connection))
Result:
CREATE OR REPLACE TRIGGER "USER"."EMP"
BEFORE INSERT OR UPDATE
of salary
on employee
for each row
declare
v_error VARCHAR2(20);
begin
if :new.salary > 10
then
v_error:=:old.first_name||' cannot have that much!';
raise_application_error(-20999,v_error);
end if;
end;
ALTER TRIGGER "USER"."EMP" ENABLE
Expected Result:
CREATE OR REPLACE TRIGGER "EMP"
BEFORE INSERT OR UPDATE
of salary
on employee
for each row
declare
v_error VARCHAR2(20);
begin
if :new.salary > 10
then
v_error:=:old.first_name||' cannot have that much!';
raise_application_error(-20999,v_error);
end if;
end;

# Won't do all the escaping for you. You need to escape the quotes around [A-Z] with '\'.
Same deal with the \1\3, '\' is saying it's an escape sequence where you really mean a literal '\'.
E.g:
OracleCommand Command = new OracleCommand(#"SELECT regexp_replace(dbms_metadata.get_ddl('TRIGGER','" + triggernames + "'),'(CREATE OR REPLACE TRIGGER )(\"[A-Z]+\"\.)(.+)(ALTER TRIGGER .+)','\\1\\3', 1, 0, 'n')FROM dual", conn1))
Alternatively if you want to use a 2nd '#' you need to use the quote-escape-sequence (a double "")
e.g.:
#"SELECT regexp_replace(dbms_metadata.get_ddl('TRIGGER','" + triggernames + #"'),'(CREATE OR REPLACE TRIGGER )(""[A-Z]+""\.)(.+)(ALTER TRIGGER .+)','\1\3', 1, 0, 'n')FROM dual"

Use Values of Project/Properties/Settings.settings in InnoSetup script?

I want to be able to change a value in one place in my C# .NET 4.0 Project. For this I use the built in Properties/Settings.settings file, which is essentially an XML file.
As we're using InnoSetup (which is pretty common) for our software, and as the settings.settings file is the default way of storing values for a C# .NET application, I wondered if there was a way for the InnoSetup script to pull values from the settings file, or if you could point me to a script that could do this to set variables of the setup script.
EDIT:
I got the XML example running, but now I cannot use an XPath query to get the correct node. This is my script:
[Code]
{--- MSXML ---}
const
XMLFileName = '..\CCFinderWPF\Properties\Settings.settings';
XMLFileName2 = '..\CCFinderWPF\Properties\Settings.xml';
function Settings(Default: String): String;
var
XMLDoc, SeekedTopNode, SeekedNode, iNode, Sel: Variant;
Path, XPath: String;
begin
{ Load the XML File }
try
XMLDoc := CreateOleObject('MSXML2.DOMDocument.4.0');
except
RaiseException('Please install MSXML first.'#13#13'(Error ''' + GetExceptionMessage + ''' occurred)');
end;
XMLDoc.async := False;
XMLDoc.resolveExternals := false;
XMLDoc.preserveWhiteSpace := true;
XMLDoc.setProperty('SelectionLanguage', 'XPath');
XMLDoc.load(XMLFileName);
if XMLDoc.parseError.errorCode <> 0 then
RaiseException('Error on line ' + IntToStr(XMLDoc.parseError.line) + ', position ' + IntToStr(XMLDoc.parseError.linepos) + ': ' + XMLDoc.parseError.reason);
MsgBox('XML-File: ' + XMLFileName, mbInformation, MB_OK);
{ Modify the XML document }
iNode := XMLDoc.documentElement;
iNode := iNode.selectSingleNode('Setting[#Name="' + Default + '"]');
// **selectSingleNode returns null, seems that selectSingleNode with XPath doesn't work?**
MsgBox('The Node is: ' + iNode.nodeName, mbInformation, MB_OK);
SeekedNode := iNode.firstChild;
Result := SeekedNode.lastChild.text;
MsgBox('The XPath is: ' + XPath, mbInformation, MB_OK);
end;
I call this function using the Inno Setup Precompiler like this:
#define ABAppName "{code:Settings|AppName}"
The XML-file looks like this:
<?xml version='1.0' encoding='utf-8'?>
<SettingsFile xmlns="http://schemas.microsoft.com/VisualStudio/2004/01/settings" CurrentProfile="(Default)" GeneratedClassNamespace="CCFinder.Properties" GeneratedClassName="Settings">
<Profiles />
<Settings>
<Setting Name="AppName" Type="System.String" Scope="Application">
<Value Profile="(Default)">CCFinder</Value>
</Setting>
...
The goal of all this is that I can set values in my app from the Settings.settings file in my C# projects, have my hudson instance checkout my code and change this XML file for different versions of the app and for example change some values that I don't necessary know at design time. I want to be able to pull my variables from this XML file. I'm just stuck using MSXML2 here. Any help would be greatly appreciated.

Nobody knew a solution, so I solved this with a while loop. Note that this is my first Delphi code, so it might not be elegant and / or 100% correct. This is how I define the needed constants:
#define ABAppName "{code:Settings|AppName}"
#define ABAppVersion "{code:Settings|AppVersion}"
#define ABCompany "{code:Settings|CompanyName}"
#define ABAppTitle "{code:Settings|ApplicationTitle}"
#define ABAppTitlePro "{code:Settings|ApplicationTitle)}"+" "+"{code:Settings|ProVersionAppender}"
#define ABCompanyUrl "{code:Settings|CompanyUrl_de}"
#define ABContactEmailDe "{code:Settings|ContactEmail_de}"
#define ABYear "{code:Settings|Year}"
These are in a constants.iss file as plain text that I include in my setup-script with #include "constants.iss" in the beginning. This is the delphi-code that is called by the code-calls:
const
XMLFileName = '..\CCFinder\Properties\Settings.settings';
function Settings(Default: String): String;
var
XMLDoc, iNode: Variant;
Path: String;
i : Integer;
Loop : Boolean;
begin
{ Load the XML File }
try
XMLDoc := CreateOleObject('MSXML2.DOMDocument.6.0');
except
RaiseException('Please install MSXML first.'#13#13'(Error ''' + GetExceptionMessage + ''' occurred)');
end;
try
XMLDoc.async := False;
XMLDoc.resolveExternals := false;
XMLDoc.load(XMLFileName);
if XMLDoc.parseError.errorCode <> 0 then
RaiseException('Error on line ' + IntToStr(XMLDoc.parseError.line) + ', position ' + IntToStr(XMLDoc.parseError.linepos) + ': ' + XMLDoc.parseError.reason);
iNode := XMLDoc.DocumentElement;
iNode := iNode.LastChild;
Loop := True;
i := 0;
while Loop do
begin
Try
if iNode.ChildNodes[i].attributes[0].nodeValue = Default
then
begin
Result := iNode.ChildNodes[i].text;
// MsgBox('Result for Default: ' + Result + Default, mbInformation, MB_OK);
i := i+1;
Loop := False;
Break;
end
else
begin
i := i+1;
if i = 100 then Loop := false;
end;
except
Result := '';
end;
end;
except
end;
end;
The loop is limited by 100 runs, because I'm too clumsy to count the XML nodes in Delphi.
The InnoSetup Preprocessor needs to be installed.
Inno doesn't seem to accept constants in path names, therefore some values just could not be drawn from the Properties/Settings.settings file.
I didn't know that the preprocessor doesn't actually use the Delphi code to fetch the correct values to include them in the script, instead it includes the complete delphi code call in all places where the constant is used. Of course this isn't what I wanted to do, as it doesn't even work when the code is embedded into paths for example. Therefore I changed the setup script to only include one constant I needed to replace by find and replace, and that is used for pathnames and the executable filename for example.
For the rest, I wrote a .NET command-line application that does basically the same thing as the delphi code, to execute it in the Hudson before the setup is compiled on Hudson. This replaces the code-calls with the actual values needed (only except, but you should get the point):
{
XmlDocument xmldoc = new XmlDocument();
xmldoc.Load(pathToSettingsFile);
XmlNodeList list = xmldoc.GetElementsByTagName("Setting");
foreach (XmlNode node in list)
{
string keystring = node.Attributes["Name"].InnerText;
string valuestring = node.FirstChild.InnerText;
settingValues.Add(keystring,valuestring);
}
var t = File.OpenText(constantsfilename);
string text;
using (t)
{
text = t.ReadToEnd();
}
string sample = "{code:Settings|";
char endsign = '}';
while (text.Contains(sample))
{
int startindex = text.IndexOf(sample);
int endindex = text.IndexOf(endsign, startindex);
int variableIndex = startindex + sample.Length;
string variable = text.Substring(variableIndex, endindex - variableIndex);
text = text.Replace(sample + variable + endsign, newVal(variable));
}
var w = new StreamWriter(constantsfilename);
using (w)
{
w.Write(text);
w.Flush();
}
}
public static string newVal(string variable)
{
if (settingValues.ContainsKey(variable))
return settingValues[variable];
else
{
return "";
}
}
This means I now can set values in my Settings-File that can be changed with yet another command-line application in hudson for special builds, and gets automatically written to the setup script.
EDIT: Just realized that it's the Visual Studio Designer that manually transfers the settings from the settings-file to the app.config, which gets changed to ProgramName.exe.config file in the output while building. Therefore you needs to change the values in there to have it have an effect, the coe should work if you give it the correct path to the file.
EDIT2: This app.config is renamed during compilation into YourAppName.exe.config ... when you're not putting it inside the setup, the values are not initialized correctly. I extended my code to also adapt the designer code of the Settings.designer.cs file with the values from the XML and guess this will do it. One configuration file to rule them all :-)