I am trying to create new folder inside my datalake store, there is no error in code however nothing is being reflected in datalake store.
Code:
using System;
using System.Collections.Generic;
using System.Text;
using System.Threading.Tasks;
using System.IO;
using System.Threading;
using Microsoft.Azure.Management.DataLake.Store;
using Microsoft.Azure.Management.DataLake.Store.Models;
using Microsoft.Rest.Azure.Authentication;
namespace test_dlstore
{
class Program
{
private static DataLakeStoreAccountManagementClient _adlsClient;
private static DataLakeStoreFileSystemManagementClient _adlsFileSystemClient;
private static string _adlsAccountName;
private static string _subId;
private static void Main(string[] args)
{
_adlsAccountName = "mycreds#mysite.com";
_subId = "2342342-97ce-a54b2-ba6e-234234234234234";
string localFolderPath = #"C:\myfolder\"; // TODO: Make sure this exists and can be overwritten.
string localFilePath = Path.Combine(localFolderPath, "try.txt");
string remoteFolderPath = "adl://mystore.azuredatalakestore.net/myfolder";
string remoteFilePath = Path.Combine(remoteFolderPath, "try.txt");
SynchronizationContext.SetSynchronizationContext(new SynchronizationContext());
var tenant_id = "my-tenant-id";
var nativeClientApp_clientId = "1950a258-227b-4e31-a9cf-717495945fc2";
var activeDirectoryClientSettings = ActiveDirectoryClientSettings.UsePromptOnly(nativeClientApp_clientId, new Uri("urn:ietf:wg:oauth:2.0:oob"));
var creds = UserTokenProvider.LoginWithPromptAsync(tenant_id, activeDirectoryClientSettings).Result;
_adlsClient = new DataLakeStoreAccountManagementClient(creds) { SubscriptionId = _subId };
_adlsFileSystemClient = new DataLakeStoreFileSystemManagementClient(creds);
Console.WriteLine(ListAdlStoreAccounts());
Console.WriteLine(AppendToFile("adl://mystore.azuredatalakestore.net/myfolder/stage/testfile.txt", "abcdefghijklmnopqrstuvwxyz"));
CreateDirectory("adl://mystore.azuredatalakestore.net/myfolder/newdir");
Console.ReadLine();
}
// Append to file
public static string AppendToFile(string path, string content)
{
using (var stream = new MemoryStream(Encoding.UTF8.GetBytes(content)))
{
Console.WriteLine(_adlsAccountName, path, content);
Console.WriteLine(path);
Console.WriteLine(content);
_adlsFileSystemClient.FileSystem.AppendAsync(_adlsAccountName, path, stream);
return "Tried";
}
}
public static string CreateDirectory(string path)
{
_adlsFileSystemClient.FileSystem.MkdirsAsync(_adlsAccountName, path);
return "Tried Creating directory.";
}
}
}
After execution of above code, program exits with no error. The connection is being made.
Also it is displaying datalake stores that are present but not able to do anything at all over the data lake stores.
I am new to azure please help me out.
I do a demo test on my side,it works correctly on side. The following is my detail steps:
Preparation:
Registry an AD application and assign role to applcation, more details please
refer to Azure official tutorials. After that we can get tenantId, appId, secretKey from the Azure Portal.
Steps:
1.Create an C# console project
2.Referece the Microsoft.Azure.Management.DataLake.Store SDK, more details please refer to packages.config file section.
3.Add the follow code
var applicationId = "appid";
var secretKey = "secretkey";
var tenantId = "tenantid";
var adlsAccountName = "adlsAccount Name";
var creds = ApplicationTokenProvider.LoginSilentAsync(tenantId, applicationId, secretKey).Result;
var adlsFileSystemClient = new DataLakeStoreFileSystemManagementClient(creds,clientTimeoutInMinutes:60);
adlsFileSystemClient.FileSystem.Mkdirs(adlsAccountName, "/tomtest/newfolder");
4.Run the test code
5.Check from the azure portal.
Packages.config
<?xml version="1.0" encoding="utf-8"?>
<packages>
<package id="Microsoft.Azure.Management.DataLake.Store" version="2.2.0" targetFramework="net452" />
<package id="Microsoft.IdentityModel.Clients.ActiveDirectory" version="3.13.8" targetFramework="net452" />
<package id="Microsoft.Rest.ClientRuntime" version="2.3.8" targetFramework="net452" />
<package id="Microsoft.Rest.ClientRuntime.Azure" version="3.3.7" targetFramework="net452" />
<package id="Microsoft.Rest.ClientRuntime.Azure.Authentication" version="2.2.0-preview" targetFramework="net452" />
<package id="Newtonsoft.Json" version="9.0.2-beta1" targetFramework="net452" />
</packages>
Related
I am trying to find an example of uploading a file to an Azure file share or blob storage from a razor page. I use this code from the tutorial
but this copy files only to the local filesystem.
Now I found the article here on stackoverflow, but it won't work. I got the error in line
var storageAccount = CloudStorageAccount.Parse(< your creds here>);
the name "CloudStorageAccount does not exist in the current context"
I am using Visual Studio 2022, .Net Core 2.2. Any help would be much appreciated.
Initially even I got the error.
The NuGet package required for CloudStorageAccount is WindowsAzure.Storage.
But in Package Manager you can see WindowsAzure.Storage is deprecated.
You can also see the alternate package Azure.Storage.Blobs is mentioned.
Install the latest Azure.Storage.Blobs NuGet package.
When I tried to Install Azure.Storage.Blobs package, got the below error.
"System.IO.Hashing doesn't support $(TargetFramework). Consider >updating the TargetFramework to netcoreapp3.1 or later." />
The error clearly says to update and use the .NET Core latest version.
I am able to clear the issue in .Net Core 2.1 by installing the below NuGet Package.
Microsoft.Azure.Storage.DataMovement
Add the using namespace using Microsoft.Azure.Storage
My .csproj file
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>netcoreapp2.1</TargetFramework>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.AspNetCore.App" />
<PackageReference Include="Microsoft.AspNetCore.Razor.Design" Version="2.1.2" PrivateAssets="All" />
<PackageReference Include="Microsoft.Azure.Storage.DataMovement" Version="2.0.4" />
</ItemGroup>
</Project>
My Conroller.cs
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.Storage;
using Microsoft.Azure.Storage.File;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using WebAppMVC.Models;
namespace WebAppMVC.Controllers
{
public class HomeController : Controller
{
public IActionResult Index()
{
return View();
}
[HttpPost]
public async Task<IActionResult> Upload(IFormFile file)
{
if (file != null)
{
using (var stream = new MemoryStream())
{
try
{
await file.CopyToAsync(stream);
stream.Seek(0, SeekOrigin.Begin);
var filename = file.FileName;
var storageAccount = CloudStorageAccount.Parse("YOUR CREDS HERE");
var client = storageAccount.CreateCloudFileClient();
var shareref = client.GetShareReference("YOUR FILES SHARE");
var rootdir = shareref.GetRootDirectoryReference();
var fileref = rootdir.GetFileReference(filename);
await fileref.DeleteIfExistsAsync();
await fileref.UploadFromStreamAsync(stream);
return Ok(new { fileuploaded = true });
}
catch (Exception ex)
{
return BadRequest(ex);
}
}
}
else
{
return BadRequest(new { error = "there was no uploaded file" });
}
}
}
}
Say I have the following xml, and I would like to get the value of the version attribute for the single node with id of Pkg1, which is expected to be 1.2.3.
<project>
<nugets id='test'> </nugets>
<packages>
<package id='test1' version='1'/>
<package id='Pkg1' version='1.2.3'/>
<package id='Pkg1Test' version='4.5.6'/>
</packages>
</project>
Below is my attempt, but the target node is in the field of OuterXml as a string for the target node:
XmlDocument xmlDoc = new XmlDocument();
xmlDoc.LoadXml(xmlStr);
string path = "project/packages/package[#id='Pkg1']"; // target node has id=Pkg1
var targetNode = xmlDoc.SelectSingleNode(path);
Console.WriteLine($"{targetNode.OuterXml}"); // prints out the target node as string.
This seems work: string version = targetN.Attributes["version"].Value, is this the right way of getting the attribute value, and why the node is in the targetNode.OuterXml?
Here is a working code to get what you need:
private const string str = #"
<project>
<nugets id='test'> </nugets>
<packages>
<package id='test1' version='1'/>
<package id='Pkg1' version='1.2.3'/>
<package id='Pkg1Test' version='4.5.6'/>
</packages>
</project>";
private static void Test()
{
var el = XElement.Parse(str);
var packages = el.Element("packages")?
.Elements("package")
.ToList();
var package = packages?
.FirstOrDefault(x => x.Attribute("id")?.Value == "Pkg1");
var id = package?.Attribute("version")?.Value;
Console.Write(id);
}
I seem to be unable to use this library in project
Severity Code Description Project File Line Suppression State
Error CS0246 The type or namespace name 'WindowsAzure' could not be found (are you missing a using directive or an assembly reference?) ClassLibrary2 \Visual Studio 2017\Projects\ClassLibrary2\ClassLibrary2\EntityListener.cs 24 Active
using WindowsAzure.Servicebus;
I installed using nuget packet manager, and it is defined in my packages.config file. Why can I not use it?
Packages.config:
<?xml version="1.0" encoding="utf-8"?>
<packages>
<package id="WindowsAzure.ServiceBus" version="4.1.10" targetFramework="net452" />
</packages>
If your .NET project version is exactly 4.5 (not 4.5.x), you will need to fall back to WindowsAzure.ServiceBus package version 4.1.3. Moreover, this
Here is the packages.config:
<?xml version="1.0" encoding="utf-8"?>
<packages>
<package id="WindowsAzure.ServiceBus" version="4.1.3" targetFramework="net45" />
</packages>
In addition, the correct namespace to use is the following:
using Microsoft.ServiceBus.Messaging;
Find below a sample .NET 4.5 Console Application that sends a message to an Azure Service Bus Queue. Please note this is just a sample and it is not production ready code. I hope this helps.
using System;
using System.Threading.Tasks;
using Microsoft.ServiceBus.Messaging;
namespace ServiceBusSample
{
class Program
{
static void Main(string[] args)
{
const string ConnectionString = "your service bus connection string";
const string QueueName = "your service bus queue name";
string message = "Hello World!";
string sessionId = Guid.NewGuid().ToString();
SendMessage(ConnectionString, QueueName, sessionId, message).Wait();
Console.WriteLine("Press <ENTER> to exit...");
Console.ReadLine();
}
private static async Task SendMessage(string connectionString, string queueName, string sessionId, string message, string correlationId = null)
{
try
{
if (string.IsNullOrWhiteSpace(connectionString))
{
throw new ArgumentException("Service bus connection string cannot be null, empty or whitespace.");
}
if (string.IsNullOrWhiteSpace(queueName))
{
throw new ArgumentException("Service bus queue name cannot be null, empty or whitespace.");
}
if (string.IsNullOrWhiteSpace(sessionId))
{
throw new ArgumentException("Session id cannot be null, empty or whitespace.");
}
QueueClient queueClient = QueueClient.CreateFromConnectionString(connectionString, queueName);
BrokeredMessage brokeredMessage = new BrokeredMessage(message);
brokeredMessage.SessionId = sessionId;
await queueClient.SendAsync(brokeredMessage);
}
catch
{
// TODO: Handle exception appropriately (including logging)
throw;
}
}
}
}
Using Azure CosmosDB Mongo.
I have text fields in documents and subdocuments. Which I want be able to search.
Using Contains works fine on the parent document properties. But doesn't seem to look at children at all. And doesn't even return any error.
Document:
{
"TextField1": "this will be found in search",
"Comments": [{
"Comment": "amazing post, let's see if this can be foundtoo",
}, {
"Comment": "thanks",
}]
}
Search:
var postFilter = Builders<MyObject>.Filter.Where(p => p.TextField1.ToLowerInvariant().Contains(searchText.ToLowerInvariant())) |
Builders<MyObject>.Filter.Where(p => p.Comments.Any(pc => pc.Comment.ToLowerInvariant().Contains(searchText.ToLowerInvariant())));
var posts = await Posts.Find(postFilter).ToListAsync();
If I use the above code with search "found". It will return the document.
If I use it with search "foundtoo". It will not return anything.
PS: I have tried using Text and it is not supported and comes back as an error.
Cosmos Mongo Db may have not implemented whole commands as native MongoDB. I tried the code you mentioned also get the same result as you mentioned.
Using Contains works fine on the parent document properties. But doesn't seem to look at children at all. And doesn't even return any error.
Please have a try to use the follow code to do that. I test it on my side, it works correctly.
var filterResult = collection.Find(Builders<MyObject>.Filter.ElemMatch("Comments", Builders<Mydata>.Filter.Where(p=>p.Comment.ToLowerInvariant().Contains(searchText.ToLowerInvariant())))).ToList();
The following is my detail steps:
1.Create a .net project and reference MongoDB.Driver more detail please refer to packages.config.
2.Add the Mydata and MyObject class
public class MyObject
{
public string TextField1;
public Mydata[] Comments;
[JsonProperty(PropertyName = "id")]
public string Id;
}
public class Mydata
{
public string Comment;
}
3.Test with following code.
string connectionString = "connection string";
MongoClientSettings settings = MongoClientSettings.FromUrl(new MongoUrl(connectionString));
settings.SslSettings = new SslSettings { EnabledSslProtocols = SslProtocols.Tls12 };
var mongoClient = new MongoClient(settings);
var searchText = "foundtoo";
var db = mongoClient.GetDatabase("dbname");
var collection = db.GetCollection<MyObject>("collectionName");
var mydata1 = new Mydata {Comment = "Thank you"};
var mydata2 = new Mydata {Comment = "amazing post, let's see if this can be foundtoo"};
var list = new List<Mydata> {mydata1, mydata2};
Mydata[] mydatas = {mydata1,mydata2};
collection.InsertOneAsync(new MyObject
{
Id = Guid.NewGuid().ToString(),
TextField1 = "this will be found in search",
Comments = mydatas
}).Wait();
var filterResult = collection.Find(Builders<MyObject>.Filter.ElemMatch("Comments", Builders<Mydata>.Filter.Where(p=>p.Comment.ToLowerInvariant().Contains(searchText.ToLowerInvariant())))).ToList();
packages.config
<?xml version="1.0" encoding="utf-8"?>
<packages>
<package id="MongoDB.Bson" version="2.4.4" targetFramework="net461" />
<package id="MongoDB.Driver" version="2.4.4" targetFramework="net461" />
<package id="MongoDB.Driver.Core" version="2.4.4" targetFramework="net461" />
<package id="Newtonsoft.Json" version="9.0.1" targetFramework="net461" />
<package id="System.Runtime.InteropServices.RuntimeInformation" version="4.3.0" targetFramework="net461" />
</packages>
While I can get the assembly version using the following code
var assembly = typeof(App).GetTypeInfo().Assembly;
var assemblyVersion = assembly.GetCustomAttribute<AssemblyFileVersionAttribute>().Version;
I would like to retrieve the Version from Package.appxmanifest in this case 1.0.0.4
<?xml version="1.0" encoding="utf-8"?>
<Package xmlns="http://schemas.microsoft.com/appx/2010/manifest" xmlns:m2="http://schemas.microsoft.com/appx/2013/manifest">
<Identity Name="zzz" Publisher="CN=zzz" Version="1.0.0.4" />
I expected to have access to Windows.ApplicationModel, but this is not available to me
Here's what you can do to retrieve the version in code:
using Windows.ApplicationModel;
public static string GetAppVersion()
{
Package package = Package.Current;
PackageId packageId = package.Id;
PackageVersion version = packageId.Version;
return string.Format("{0}.{1}.{2}.{3}", version.Major, version.Minor, version.Build, version.Revision);
}
Reference: http://www.michielpost.nl/PostDetail_67.aspx