Querying AllegroGraph using dotNetRDF - c#

I've got a AllegroGraph server running and have trouble querying remote datastore, there's very little information on documentation.
Here's my lil piece of code:
using System;
using VDS.RDF;
using VDS.RDF.Storage;
namespace PoC {
class Program {
static void Main(string[] args) {
string server = "http://server";
string repo = "repo";
string username = "username";
string password = "password";
AllegroGraphConnector agraph = new AllegroGraphConnector(server, repo, username, password);
Options.HttpDebugging = true;
Options.HttpFullDebugging = true;
agraph.Query("SELECT * WHERE { emid:_PCAT_0001 ?p ?o }");
Console.ReadLine();
}
}
}
MALFORMED QUERY: Parse error: namespace mapping for "emid" not defined
when expanding QName "emid:_PCAT_0001".
Although in AllegroGraph WebView I can run exactly the same query and namespace is loaded into repository.
How do I resolve that?

You will need to declare the prefix emid: in your query. Presumably the AllegroGraph WebView UI is doing that automatically for you, but the plain SPARQL endpoint won't.
Try using something like this:
agraph.Query("PREFIX emid: <http://your.uri.goes/here> SELECT * WHERE { emid:_PCAT_0001 ?p ?o }");
Obviously you should replace that fake URI with the real URI that your emid: prefix maps to!

Related

How can I retrieve the machine name from a remote Windows computer?

I am trying to find a way to retrieve Computer name from an IP address in C#, but all of the answers marked online as retrieving machine name or computer name actually get the hostname, not the Computer Name. If you go to Control panel > system, there is a property in that menu "Computer name" ... I'm looking for this value on a remote machine. AFAIK, the HOSTNAME will = the full computer name IF there is no DNS mapping. The problem is that these servers I'm working on do have DNS mapping so the hostname returns their DNS address.
Feel free to correct me on the technical details of that if I said anything wrong, but the question will remain.
I've tried this:
IPHostEntry hostEntry = Dns.GetHostEntry(_ip);
_hostname = hostEntry.HostName;
but obviously returns the HostName, not the Computer name. I could also settle for the "Full computer name" property being returned, and then simply strip the unneeded parts of the string off to reveal the "Computer name."
Also, if you know how to do this with PowerShell, I could use your help as-well. I am hosting the PowerShell engine in my app anyway... so could simply pass your command into PowerShellInstance.AddScript(_yourCommandHere); and pipe its return back into my app.
Please advise if this is possible to do.
#DanielAWhite
EDIT: How is this a duplicate of the listed answer? The answer in that post says exactly what I posted as being the problem to this question. No, that's not a duplicate, because I'm NOT looking for the hostname. I specifically told you in my OP that I wasn't looking for that, and they're not asking what I'm asking. If there is no way to get the computer name from the IP in .NET then just answer the question with that.
From the "duplicate":
Well, not every IP address has a name. However, given the IPAddress you can use >Dns.GetHostEntry to try to resolve it. Also note that if it's being a NAT >router, you'll be getting the router's IP address rather than their actual >machine.
look at my OP... .GetHostEntry DOESN'T work. that's the whole reason I took the time to type this up.
thanks
DOUBLE EDIT: BACON has an answer for how to do this; this post was locked because someone didn't take the time to actually read what I wrote. Since it's locked, you also can't give a better answer. But here's how I did it, saving this here for future reference:
//declare a string to be our machinename
string machineName;
//declare a string which we will pass into powershell later as script
//assigns the hostname or IP
string getComputer = "$ip = " + "\"" + ip + "\"" + "\r\n";
//add to the string this, which gets the Win32_ComputerSystem.. #BACON knew what I was after
//we pipe that back using |select -expand Name
getComputer += "get-wmiobject -class Win32_ComputerSystem -property Name -ComputerName " + "$ip " +
"|select -expand Name";
//create a powershell instance using
using (PowerShell PowerShellInstance = PowerShell.Create())
{
//add the script into our instance of ps
PowerShellInstance.AddScript(getComputer);
//instantiate a collection to house our output from PS
//you could also probably just instantiate a PSObject instead of a collection.. but this might be useful if modified to get an array of computer names... and this is how I did it so can't verify
Collection<PSObject> psOutput;
//assign psOutput from .Invoke() method
psOutput = PowerShellInstance.Invoke();
//you could trim this loop and get rid of it for only one IP
foreach (var item in psOutput)
{
//machineName = MachineName||ComputerName string NOT hostname
machineName = item.BaseObject.ToString();
}
}
Oh, and per bacon in the comments, you have to have WMI allowed through windows firewall for this to work. It worked perfectly for me.
Reconstituting my comments as an answer...
Imagine we had an interface like this...
namespace SO56585341
{
public interface IComputerInfoSource
{
string GetComputerName();
}
}
There are a handful of ways to implement this to get the machine name of the local computer. The simplest is to return the value of the Environment.MachineName property...
namespace SO56585341
{
public class EnvironmentClassComputerInfoSource : IComputerInfoSource
{
public string GetComputerName()
{
return System.Environment.MachineName;
}
}
}
You can also use the Environment.GetEnvironmentVariable() method to retrieve the value of the %ComputerName% environment variable...
namespace SO56585341
{
public class EnvironmentVariableComputerInfoSource : IComputerInfoSource
{
public string GetComputerName()
{
return System.Environment.GetEnvironmentVariable("ComputerName");
}
}
}
You can p/invoke the GetComputerName() Windows API function, which is what Environment.MachineName does behind the scenes...
using System.Runtime.InteropServices;
using System.Text;
namespace SO56585341
{
public class WinApiComputerInfoSource : IComputerInfoSource
{
private const int MAX_COMPUTERNAME_LENGTH = 15;
[DllImport("Kernel32.dll", CallingConvention = CallingConvention.Winapi, CharSet = CharSet.Auto, SetLastError = true)]
private static extern bool GetComputerName(
StringBuilder lpBuffer,
ref int nSize
);
public string GetComputerName()
{
int maxCapacity = MAX_COMPUTERNAME_LENGTH + 1;
StringBuilder nameBuilder = new StringBuilder(maxCapacity, maxCapacity);
if (!GetComputerName(nameBuilder, ref maxCapacity))
{
// TODO: Error handling...
throw new System.ComponentModel.Win32Exception();
}
return nameBuilder.ToString();
}
}
}
You can use WMI to retrieve the Name property of the singleton Win32_ComputerSystem class. You can do this by instantiating a ManagementClass instance for the Win32_ComputerSystem class and calling GetInstances() on it to retrieve an array containing the sole instance...
using System.Linq;
using System.Management;
namespace SO56585341
{
public class WmiClassComputerInfoSource : IComputerInfoSource
{
public string GetComputerName()
{
using (ManagementClass computerSystemClass = new ManagementClass("Win32_ComputerSystem"))
using (ManagementObjectCollection computerSystemCollection = computerSystemClass.GetInstances())
using (ManagementObject computerSystem = computerSystemCollection.Cast<ManagementObject>().Single())
return (string) computerSystem["Name"];
}
}
}
...or by creating a ManagementObjectSearcher and using that to Get() the lone Win32_ComputerSystem instance...
using System.Linq;
using System.Management;
namespace SO56585341
{
public class WmiSearcherComputerInfoSource : IComputerInfoSource
{
public string GetComputerName()
{
ObjectQuery computerSystemQuery = new SelectQuery("Win32_ComputerSystem");
using (ManagementObjectSearcher computerSystemSearcher = new ManagementObjectSearcher(computerSystemQuery))
using (ManagementObjectCollection computerSystemCollection = computerSystemSearcher.Get())
using (ManagementObject computerSystem = computerSystemCollection.Cast<ManagementObject>().Single())
return (string) computerSystem["Name"];
}
}
}
Finally, the value returned by all of the methods above seems to ultimately be stored in the registry, so if you don't mind relying on that implementation detail you can retrieve it from there directly...
using Microsoft.Win32;
namespace SO56585341
{
public class RegistryComputerInfoSource : IComputerInfoSource
{
public string GetComputerName()
{
// See also #"SYSTEM\CurrentControlSet\Control\ComputerName\ActiveComputerName\"
// https://www.oreilly.com/library/view/windows-nt-workstation/9781565926134/10_chapter-07.html
const string valueParentKeyPath = #"SYSTEM\CurrentControlSet\Control\ComputerName\ComputerName\";
using (RegistryKey parentKey = Registry.LocalMachine.OpenSubKey(valueParentKeyPath, false))
return (string) parentKey.GetValue("ComputerName");
}
}
}
As for getting the same value from a remote computer only the last three implementations above will work, though with minimal tweaking required. First, just to complete this IComputerInfoSource example, let's create an abstract class to hold the remote machine name/address "parameter"...
namespace SO56585341
{
public abstract class RemoteComputerInfoSource : IComputerInfoSource
{
public string RemoteNameOrIp
{
get;
}
protected RemoteComputerInfoSource(string nameOrIp)
{
RemoteNameOrIp = nameOrIp ?? throw new System.ArgumentNullException(nameof(nameOrIp));
}
public abstract string GetComputerName();
}
}
Retrieving the Win32_ComputerSystem instance via a ManagementClass just becomes a matter of explicitly passing it a ManagementPath that also specifies the NamespacePath and Server...
using System.Linq;
using System.Management;
namespace SO56585341
{
public class RemoteWmiClassComputerInfoSource : RemoteComputerInfoSource
{
public RemoteWmiClassComputerInfoSource(string nameOrIp)
: base(nameOrIp)
{
}
public override string GetComputerName()
{
ManagementPath computerSystemPath = new ManagementPath() {
ClassName = "Win32_ComputerSystem",
NamespacePath = #"root\cimv2",
Server = RemoteNameOrIp
};
using (ManagementClass computerSystemClass = new ManagementClass(computerSystemPath))
using (ManagementObjectCollection computerSystemCollection = computerSystemClass.GetInstances())
using (ManagementObject computerSystem = computerSystemCollection.Cast<ManagementObject>().Single())
return (string) computerSystem["Name"];
}
}
}
A ManagementObjectSearcher can be used by passing a similar ManagementPath wrapped in a ManagementScope...
using System.Linq;
using System.Management;
namespace SO56585341
{
public class RemoteWmiSearcherComputerInfoSource : RemoteComputerInfoSource
{
public RemoteWmiSearcherComputerInfoSource(string nameOrIp)
: base(nameOrIp)
{
}
public override string GetComputerName()
{
ManagementScope computerSystemScope = new ManagementScope(
new ManagementPath() {
NamespacePath = #"root\cimv2",
Server = RemoteNameOrIp
}
);
ObjectQuery computerSystemQuery = new SelectQuery("Win32_ComputerSystem");
using (ManagementObjectSearcher computerSystemSearcher = new ManagementObjectSearcher(computerSystemScope, computerSystemQuery))
using (ManagementObjectCollection computerSystemCollection = computerSystemSearcher.Get())
using (ManagementObject computerSystem = computerSystemCollection.Cast<ManagementObject>().Single())
return (string) computerSystem["Name"];
}
}
}
Querying a remote registry just requires an additional call to OpenRemoteBaseKey() to get a handle to the root of the remote hive...
using Microsoft.Win32;
namespace SO56585341
{
public class RemoteRegistryComputerInfoSource : RemoteComputerInfoSource
{
public RemoteRegistryComputerInfoSource(string nameOrIp)
: base(nameOrIp)
{
}
public override string GetComputerName()
{
// See also #"SYSTEM\CurrentControlSet\Control\ComputerName\ActiveComputerName\"
// https://www.oreilly.com/library/view/windows-nt-workstation/9781565926134/10_chapter-07.html
const string valueParentKeyPath = #"SYSTEM\CurrentControlSet\Control\ComputerName\ComputerName\";
using (RegistryKey baseKey = RegistryKey.OpenRemoteBaseKey(RegistryHive.LocalMachine, RemoteNameOrIp))
using (RegistryKey parentKey = baseKey.OpenSubKey(valueParentKeyPath, false))
return (string) parentKey.GetValue("ComputerName");
}
}
}
If you compile all of the above code into a project you can use the following Program class to test it...
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
namespace SO56585341
{
public static class Program
{
private const string TestHost = "127.0.0.1";
public static void Main()
{
// Get all non-abstract classes in the executing assembly that implement IComputerInfoSource
IEnumerable<Type> computerInfoSourceTypes = Assembly.GetExecutingAssembly().GetTypes()
.Where(type => type.IsClass && !type.IsAbstract && typeof(IComputerInfoSource).IsAssignableFrom(type));
// For each constructor in each candidate class...
foreach (Type computerInfoSourceType in computerInfoSourceTypes)
foreach (ConstructorInfo constructor in computerInfoSourceType.GetConstructors())
{
ParameterInfo[] constructorParameters = constructor.GetParameters();
object[] instanceParameters;
// If the constructor takes no parameters...
if (!constructorParameters.Any())
instanceParameters = Array.Empty<object>();
// ...or a single string parameter...
else if (constructorParameters.Length == 1 && constructorParameters[0].ParameterType == typeof(string))
instanceParameters = new object[1] { TestHost };
// ...otherwise skip this constructor
else
continue;
// Instantiate the class using the constructor parameters specified above
IComputerInfoSource computerInfoSource = (IComputerInfoSource) constructor.Invoke(instanceParameters);
string result;
try
{
result = computerInfoSource.GetComputerName();
}
catch (Exception ex)
{
result = ex.ToString();
}
Console.WriteLine(
"new {0}({1}).{2}(): \"{3}\"",
computerInfoSourceType.Name,
string.Join(
", ",
instanceParameters.Select(value => $"\"{value}\"")
),
nameof(IComputerInfoSource.GetComputerName),
result
);
}
}
}
}
I found this code to work whether TestHost was set to a machine name, CNAME, or IP address. Note that the Remote*ComputerInfoSource classes will fail if...
The appropriate service (RemoteRegistry or Winmgmt) is not running on the remote machine, or...
The appropriate firewall rule (e.g. WMI-WINMGMT-In-TCP) is not enabled on the remote machine, or...
The code is not run as a user with privileges to access the remote service.
As for PowerShell, one should be able to port the code of any of the above methods from C# (either a direct translation or using PowerShell's conveniences) and wrap them in a call to Invoke-Command since that code will be executed local to the remote machine. For example...
Invoke-Command -ComputerName $nameOrIp -ScriptBlock { $Env:COMPUTERNAME }
...or...
Invoke-Command -ComputerName $nameOrIp -ScriptBlock {
# See also 'HKLM:\SYSTEM\CurrentControlSet\Control\ComputerName\ActiveComputerName\'
# https://www.oreilly.com/library/view/windows-nt-workstation/9781565926134/10_chapter-07.html
Get-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Control\ComputerName\ComputerName\' -Name 'ComputerName'
}
PowerShell also has the Get-WmiObject...
Get-WmiObject -Class 'Win32_ComputerSystem' -ComputerName $nameOrIp -Property 'Name'
...and Get-CimInstance cmdlets...
Get-CimInstance -Class 'Win32_ComputerSystem' -ComputerName $nameOrIp -Property 'Name'
...that make working with WMI much easier. In general, I would recommend using WMI since it is pretty easy to use from C# and PowerShell for both local and remote queries, and it exists for exactly this purpose of retrieving system details without having to know about the underlying API calls or data representation.
Note that when using the Invoke-Command or Get-CimInstance cmdlets that the WinRM service must be running on the remote machine and the appropriate firewall rule (e.g. WINRM-HTTP-In-TCP-NoScope) must be enabled. Also, when passing an IP address to the -ComputerName parameter of either of those cmdlets that address must be matched by the value of WSMan:\localhost\Client\TrustedHosts. If you need to scan an entire network by IP address I tested and found that TrustedHosts accepts the * wildcard but not subnet masks, CIDR notation, or the ? wildcard.

Rocketr IPN with C# ASP.NET MVC

So i am fiddling with this website's IPN function to see if i wan't to incorporate it to some dumb project my friends and i are working on. To be honest i don't know much C# in depth, i'm fairly new to the language (a few months of coding practice).
This is the PHP sample they give out on how to use it:
https://github.com/Rocketr/rocketrnet-ipn-php/blob/master/example_rocketr_ipn.php
I am trying to make a receiver like that in MVC 5. I have the Model setup with a function when the IPN hits the server page to process the request but it seems to just fail out everytime and not write any raw data i am trying to capture to the logs.
// GET: Purchases/Incoming
public void Incoming()
{
var ipnDebugLog = HttpRuntime.AppDomainAppPath + "/Logs/IPN/debug.txt";
var testIPNKey = "the hash here";
byte[] ipnToByes = Encoding.ASCII.GetBytes(testIPNKey); // IPN string to byteto hash with
var recvdIPN = Request["HTTP_IPN_HASH"];
HMACSHA256 testHash = new HMACSHA256(ipnToByes); // Setting testHash to IPN secret string
string ipnHeader = Request["IPN_HASH"];
using (StreamWriter sw = new StreamWriter(ipnDebugLog))
{
sw.WriteLine(ipnHeader);
foreach (var reqHead in ipnHeader)
{
sw.WriteLine(reqHead.ToString());
sw.WriteLine("JSON String: " + Request["HTTP_IPN_SECRET"]);
sw.WriteLine(recvdIPN);
sw.WriteLine("From: " + GetIPAddress());
}
}
}
So this is just me trying to get the data being sent from Rocketr. On the site it states:
To verify the integrity of the payload, we will send an HMAC signature
over a HTTP header called “IPN_HASH”. The HMAC signature will be a
signed json encoded POST object with your IPN secret. You can see how
to verify the signature in the example_rocketr_ipn.php file in this
repository.
Am i just to dumb and new to understand C# to function like this? I feel like i'm on the right track to reading the raw data but i'm probly wrong?
So to sum up the question
Am i doing the incorrect way to read a raw custom HTTP header called IPN_HASH? Going off of the PHP example they used isset to read a server variable header labled HTTP_IPN_HASH right?
So i have to convert this $hmac = hash_hmac("sha512", json_encode($_POST), trim($IPN_SECRET));
Try this (make adjustments as needed/necessary):
using System;
using System.Linq;
using System.Security.Cryptography;
using System.Text;
using System.Web;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Newtonsoft.Json;
namespace Foo
{
[TestClass]
public class UnitTest1
{
[TestMethod]
public void PhpHashTest()
{
string IPN_SECRET = "I-am-the-secret";
//Mocking some HTTP POSTed data
var someFormUrlEncodedData = "order_id=1234&product_title=Hello%20World&product_id=Sku123";
//Mocking json_encode($_POST)
var data = HttpUtility.ParseQueryString(someFormUrlEncodedData);
var dictionary = data.AllKeys.ToDictionary(key => key, key => data[key]);
//{"order_id":"1234","product_title":"Hello World","product_id":"Sku123"}
var json = JsonConvert.SerializeObject(dictionary);
byte[] bytes;
using (var hmac512 = new HMACSHA512(Encoding.ASCII.GetBytes(IPN_SECRET)))
{
bytes = hmac512.ComputeHash(Encoding.ASCII.GetBytes(json));
}
//will contain lower-case hex like Php hash_hmac
var hash = new StringBuilder();
Array.ForEach(bytes, b => hash.Append(b.ToString("x2")));
//Assert that Php output exactly matches implementation
Assert.IsTrue(string.Equals("340c0049bde54aa0d34ea180f8e015c96edfc4d4a6cbd7f139d80df9669237c3d564f10366f3549a61871779c2a20d2512c364ee56af18a25f70b89bd8b07421", hash.ToString(), StringComparison.Ordinal));
Console.WriteLine(hash);
}
}
}
Not a PHP dev - this is my "Php version":
<?php
$IPN_SECRET = 'I-am-the-secret';
# 'order_id=1234&product_title=Hello%20World&product_id=Sku123';
$json = array('order_id' => '1234', 'product_title' => 'Hello World', 'product_id' =>'Sku123');
echo json_encode($json);
echo "<br />";
$hmac = hash_hmac("sha512", json_encode($json), trim($IPN_SECRET));
echo $hmac;
?>
Hth....

XmlReader testing of string returned for newline

I am writing a C# class to use for generating email lists to use when a process either succeeds or fails. In running through XmlReader examples from the web, I found that validating the Read() is tougher than it looks.
I can use string.IsNullOrEmpty(x) to test for a null value or and empty node, but it will still blow by that test showing a "\n " in the tooltip for x. Testing for "\n ", '\n ', '\n'. "\n" or char(13) all fail. If I use x.Contains((char)13), it always find it and goes into the code trying to build the email address list. So far, it either always fails or always succeeds.
I found some old posts on stackoverflow where it seemed like the question was the same, but my results don't match with the answers. My environment is Windows 8.1 running Visual Studio 2013 with .Net Framework 4.51. The example from the web I was trying to make work before using the solution in my class is at Microsoft.com
My conversion is below:
using System;
using System.Text;
using System.Linq;
using System.Xml;
using System.Xml.Schema;
using System.IO;
using System.Collections.Generic;
namespace XMLDemo
{
public class project
{
public static void Main()
{
string uri = #"C:\\events\items.xml";
string process_state = "Item";
string emails = StreamEmailAddress(uri, process_state);
}
private static string StreamEmailAddress(string uri, string process_state)
{
XmlReaderSettings settings = new XmlReaderSettings();
settings.DtdProcessing = DtdProcessing.Parse;
XmlReader reader = XmlReader.Create(uri, settings);
string returnValue = "";
reader.MoveToContent();
while (reader.Read())
{
string x = reader.Value;
if ((string.IsNullOrEmpty(x) == false) && (x.Contains((char)13)))
{
returnValue = returnValue + x + "; ";
}
}
Console.WriteLine("Made it to the end: " + returnValue);
return returnValue;
}
}
}
You should use string.IsNullOrWhiteSpace
SOLVED: After futzing with it all day, I looked at it with slightly fresher eyes after posting and saw the blatant error. I was using the incorrect function and trying to resolve line feeds by myself. By replacing IsNullOrEmpty(x) with IsNullOrWhiteSpace(x), I got the string of data as expected. Doing the code with email addresses will be easy now.

Submit C# MapReduce Job Windows Azure HDInsight - Response status code does not indicate success: 500 (Server Error)

I'm trying to submit a MapReduce job to HDInsight cluster. In my job I didn't write reduce portion because I don't want to reduce anything. All I want to do is to parse the each filename and append the values to every line in the file. So that I will have all the data needed inside the file.
My code is
using Microsoft.Hadoop.MapReduce;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace GetMetaDataFromFileName
{
class Program
{
static void Main(string[] args)
{
var hadoop = connectAzure();
//Temp Workaround to Env Variables
Environment.SetEnvironmentVariable("HADOOP_HOME", #"c:\hadoop");
Environment.SetEnvironmentVariable("Java_HOME", #"c:\hadoop\jvm");
var result = hadoop.MapReduceJob.ExecuteJob<MetaDataGetterJob>();
}
static IHadoop connectAzure()
{
//TODO: Update credentials and other information
return Hadoop.Connect(
new Uri("https://sampleclustername.azurehdinsight.net//"),
"admin",
"Hadoop",
"password",
"blobstoragename.blob.core.windows.net", //Storage Account that Log files exists
"AccessKeySample", //Storage Account Access Key
"logs", //Container Name
true
);
}
//Hadoop Mapper
public class MetaDataGetter : MapperBase
{
public override void Map(string inputLine, MapperContext context)
{
try
{
//Get the meta data from name of the file
string[] _fileMetaData = context.InputFilename.Split('_');
string _PublicIP = _fileMetaData[0].Trim();
string _PhysicalAdapterMAC = _fileMetaData[1].Trim();
string _BootID = _fileMetaData[2].Trim();
string _ServerUploadTime = _fileMetaData[3].Trim();
string _LogType = _fileMetaData[4].Trim();
string _MachineUpTime = _fileMetaData[5].Trim();
//Generate CSV portion
string _RowHeader = string.Format("{0},{1},{2},{3},{4},{5},", _PublicIP, _PhysicalAdapterMAC, _BootID, _ServerUploadTime, _LogType, _MachineUpTime);
//TODO: Append _RowHeader to every row in the file.
context.EmitLine(_RowHeader + inputLine);
}
catch(ArgumentException ex)
{
return;
}
}
}
//Hadoop Job Definition
public class MetaDataGetterJob : HadoopJob<MetaDataGetter>
{
public override HadoopJobConfiguration Configure(ExecutorContext context)
{
//Initiate the job config
HadoopJobConfiguration config = new HadoopJobConfiguration();
config.InputPath = "asv://logs#sample.blob.core.windows.net/Input";
config.OutputFolder = "asv://logs#sample.blob.core.windows.net/Output";
config.DeleteOutputFolder = true;
return config;
}
}
}
}
Usually what do you thing the reason of 500 (Server Error) ? Am I suppling to wrong credentials ? Actually I didn't really understand the difference between Username and HadoopUser parameters in Hadoop.Connect method ?
Thank you,
I had approximately same issue in the past (was unable to submit hive job to the cluster with BadGateway response). I have contacted the support team and in my case the problem was in memory leakage at the head node, what means that the problem was not at client's side and it seems to be inherited hadoop problem.
I've solved that stuff by redeploying the cluster.
Have you tried to submit other jobs (simple ones)? If so, than I suggest to have a contact with azure support team or just redeploy the cluster if it's not painful for you.

.NET Application for XML and SQL Server Integration

As a database developer with a very small amount of programming experience, I'm currently attempting to develop a C# .NET console application to import XML files into a SQL Server database. Once the import is done, I need to create a 'Response' file to be passed back to another application.
In doing my own research, I've come across the SQLBULKLOAD class. In fact, I've found some example code online that shows (at least partially) exactly what I'm trying to do:
using System;
using System.IO;
using System.Collections;
using SQLXMLBULKLOADLib;
using System.Data.OleDb;
using System.Diagnostics;
namespace SQLXmlExample
{
class Program
{
[STAThread]
static void Main(string[] args)
{
string schema = "C:\\ImportSample\\MappingFile.xml";
string datafile = "C:\\ImportSample\\DataFile.xml";
string connectionString = #"provider=SQLOLEDB;data source=localhost;database=SqlXmlDemo;Integrated Security=SSPI;";
for (int i = 0; i < args.Length; i++)
{
switch (args[i].ToLower())
{
case "-schema":
schema = args[i + 1];
break;
case "-datafile":
datafile = args[i + 1];
break;
}
}
if (schema == string.Empty || datafile == string.Empty)
{
Console.WriteLine("Missing Schema or Data File. Format: SqlXmlExample -datafile [filename] -schema [filename]");
return;
}
Load(datafile, schema, connectionString);
}
static public void Load(string XMLFilename, string XMLMappingFilename, string ConnectionString)
{
SQLXMLBULKLOADLib.SQLXMLBulkLoad loader = new SQLXMLBULKLOADLib.SQLXMLBulkLoad();
loader.CheckConstraints = true;
loader.XMLFragment = true;
loader.SchemaGen = true;
loader.SGDropTables = false;
loader.Transaction = false;
loader.ConnectionString = ConnectionString;
loader.Execute("C:\\ImportSample\\MappingFile.xml", "C:\\ImportSample\\DataFile.xml");
}
}
}
With the code above, I'm able to import an .XML file into a SQL Server instance. However, my problem is generating the 'Response' .XML file to provide information about the bulk loading operation (i.e. how many records were inserted, if the insert was successful).
As it stands now, I'm thinking that I may be using the wrong class for what I'm trying to accomplish. Am I using the correct class, or should I be using a different one?
If possible, could anyone point me in the direction of some more material to assist me? Any help would be greatly appreciated.
How about using the ErrorLogFile?
This is where the bulk loader stores all its errors and messages, simply try one out, make it fail, see what the format is and then you can then load, parse and generate xml?

Categories