We Keep Coding

Get List of all files from all folder from Azure DataLake using C#

Can we have some example to get list of all files from all folder from Azure Data Lake using .NET(C#).
we are doing in Data factory lookup activity but performance is not good.
we need to check a alternate way to get list of file and write in log folder
Blockquote

Here is how it worked for me
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Auth;
using Microsoft.WindowsAzure.Storage.Blob;
using System;
using System.Collections.Generic;
using System.IO;
using System.Text;
using System.Threading.Tasks;
namespace ConsoleApp1
{
class Program
{
static async Task Main(string[] args)
{
var account = new CloudStorageAccount(new StorageCredentials("<YOUR ACCOUNT NAME>", "<YOUR CONNECTION STRING>"), true);
var containerName = "<YOUR CONTAINER NAME>";
var blobClient = account.CreateCloudBlobClient();
var container = blobClient.GetContainerReference(containerName);
BlobContinuationToken token = null;
do
{
var blobPrefix = "";
var useFlatBlobListing = true;
var blobsListingResult = container.ListBlobsSegmentedAsync(blobPrefix, useFlatBlobListing, BlobListingDetails.None, 500, token, null, null);
var blobsList = blobsListingResult.Result;
foreach (var item in blobsList.Results)
{
var blobName = (item as CloudBlob).Name;
Console.WriteLine(blobName);
}
}
while (token != null);
}
}
}
OUTPUT :
REFERENCES
How to list all virtual directories and subdirectories

Nethereum C# how can I get balance in address?

I try to get balance in address.
It is my code:
`using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Nethereum.Web3;
using System.Threading.Tasks;
namespace ConsoleApp1
{
class Program
{
static void Main(string[] args)
{
Bananas().Wait();
}
static private async Task Bananas()
{
var publicKey = "0xC0b4ec83028307053Fbe8d00ba4372384fe4b52B";
var web3 = new Nethereum.Web3.Web3("https://ropsten.infura.io/myInfura");
//var txCount = await web3.Eth.Transactions.GetTransactionCount.SendRequestAsync(publicKey);
var balance = await web3.Eth.GetBalance.SendRequestAsync(publicKey);
var etherAmount = Web3.Convert.FromWei(balance.Value);
Console.WriteLine(web3);
Console.WriteLine("Get txCount ", etherAmount);
Console.ReadLine();
}
}
}`
I installed Nethereum via PM console: Nethereum.
I use a normal link of infura.
Why don`t I get a next result without the balance in address?

I have just made a console application with your code and all the data is coming back fine from Nethereum.
Your issue is with your Console.WriteLine(... You are passing the etherAmount as a arg0 property into the Console.WriteLine which will not output correctly on the console when you run it.
Try this
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Nethereum.Web3;
using System.Threading.Tasks;
namespace ConsoleApp1
{
class Program
{
static void Main(string[] args)
{
Bananas().Wait();
}
static private async Task Bananas()
{
var publicKey = "0xC0b4ec83028307053Fbe8d00ba4372384fe4b52B";
var web3 = new Nethereum.Web3.Web3("https://ropsten.infura.io/myInfura");
//var txCount = await web3.Eth.Transactions.GetTransactionCount.SendRequestAsync(publicKey);
var balance = await web3.Eth.GetBalance.SendRequestAsync(publicKey);
var etherAmount = Web3.Convert.FromWei(balance.Value);
Console.WriteLine(web3);
Console.WriteLine("Get txCount " + etherAmount);
Console.ReadLine();
}
}
}
ps nice 1110.337873197299357846 ETH ;) (i know it is only test ETH but we can dream)

Xamarin.Forms ios WebViewRenderer sharing Cookies

I'm using a WebViewRenderer to setup the cookie policy and also to share cookies from a login request from an HTTPClient. It turns out that as much as I give the set:
var cookieJar = NSHttpCookieStorage.SharedStorage;
cookieJar.AcceptPolicy = NSHttpCookieAcceptPolicy.Always;
When debugging in the iphone simulator and running the webview it is indicated in the browser that the cookies policy is not enabled, so the user can not log in because the webview runs an iframe from a secure environment. Below is the code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Foundation;
using UIKit;
using Xamarin.Forms;
using Projeto.Custom;
using Projeto.iOS.Renderers;
using Xamarin.Forms.Platform.iOS;
using WebKit;
using System.IO;
using System.Net;
[assembly: ExportRenderer(typeof(CustomWebView), typeof(CustomWebViewRenderer))]
namespace Mynamespace.iOS.Renderers
{
public class CustomWebViewRenderer : ViewRenderer<CustomWebView, WKWebView>
{
protected override void OnElementChanged(ElementChangedEventArgs<CustomWebView> e)
{
base.OnElementChanged(e);
if(Control == null)
{
var userController = new WKUserContentController();
var config = new WKWebViewConfiguration {
UserContentController = userController };
var webView = new WKWebView(Frame, config);
SetNativeControl(webView);
}
if(e.OldElement != null)
{
var hybrid = e.OldElement as CustomWebView;
hybrid.Cleanup();
}
if(e.NewElement != null)
{
var baseUrl = new NSUrl(NSBundle.MainBundle.BundlePath,
true);
string content = Element.Uri;
Control.LoadHtmlString(content, baseUrl);
var cookieUrl = new
Uri("https://secure.gooddata.com/gdc/account/login");
var cookieJar = NSHttpCookieStorage.SharedStorage;
cookieJar.AcceptPolicy = NSHttpCookieAcceptPolicy.Always;
foreach (var aCookie in cookieJar.Cookies)
{
cookieJar.DeleteCookie(aCookie);
}
var jCookies =
CustomCookie.CookieContainer.GetCookies(cookieUrl);
IList<NSHttpCookie> eCookies =
(from object jCookie in jCookies
where jCookie != null
select (Cookie)jCookie
into netCookie
select new NSHttpCookie(netCookie)).ToList();
cookieJar.SetCookies(eCookies.ToArray(), cookieUrl, cookieUrl);
}
}
}
}
If anyone can tell me the best way to enable cookie policy in ios native WebView, I will be grateful.

Generating msi transform using c#

I am creating a cutomization software which will do all the standardization to a mst file.
Below is the code of class that will change product name and genrate transform.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using WindowsInstaller;
using System.Data;
namespace Automation
{
class CustomInstaller
{
public CustomInstaller()
{
}
public Record getInstaller(string msiFile,MsiOpenDatabaseMode mode,string query)
{
Type type = Type.GetTypeFromProgID("WindowsInstaller.Installer");
Installer inst = (Installer)Activator.CreateInstance(type);
Database db = inst.OpenDatabase(msiFile, mode);
WindowsInstaller.View view = db.OpenView(query);
view.Execute(null);
Record record = view.Fetch();
db.Commit();
return record;
}
public bool generateTrans(string file1, string file2,string transName)
{
Type type = Type.GetTypeFromProgID("WindowsInstaller.Installer");
Installer inst = (Installer)Activator.CreateInstance(type);
Database db1 = inst.OpenDatabase(file1, MsiOpenDatabaseMode.msiOpenDatabaseModeReadOnly);
try
{
Database db2 = inst.OpenDatabase(file2, MsiOpenDatabaseMode.msiOpenDatabaseModeReadOnly);
return db2.GenerateTransform(db1, transName);
}
catch (Exception e) { }
return false;
}
public int editTransform(string msiFile, MsiOpenDatabaseMode mode, string query)
{
Type type = Type.GetTypeFromProgID("WindowsInstaller.Installer");
Installer inst = (Installer)Activator.CreateInstance(type);
Database db = inst.OpenDatabase(msiFile, mode);
WindowsInstaller.View view = db.OpenView(query);
view.Execute(null);
db.Commit();
int o=(int)db.DatabaseState;
db = null;
inst = null;
type = null;
return 1;
}
}
}
First editTransform() is called which will create a copy of original msi and do some changes in it, then generateTrans() is called which will get difference detween two msi files and create a transform file.
Now issue is when genrateTrans() is called, then it goes to catch block of it as inst.OpenDatabase return "MSI Api Error".
It seems to me that the copy of file crated by editTransform is still locked by it and is not available for use for generateTrans() menthod.
Please help here.
PS: mode used for edit transform is transact.

Instead of doing the COM Interop, checkout the far superior interop library ( Microsoft.Deployment.WindowsInstaller ) found in Windows Installer XML Deployment Tools Foundation. You'll find it much easier to use.
using System;
using System.IO;
using Microsoft.Deployment.WindowsInstaller;
namespace ConsoleApplication1
{
class Program
{
const string REFERENCEDATABASE = #"C:\orig.msi";
const string TEMPDATABASE = #"C:\temp.msi";
const string TRANSFORM = #"c:\foo.mst";
static void Main(string[] args)
{
File.Copy(REFERENCEDATABASE, TEMPDATABASE, true);
using (var origDatabase = new Database(REFERENCEDATABASE, DatabaseOpenMode.ReadOnly))
{
using (var database = new Database(TEMPDATABASE, DatabaseOpenMode.Direct))
{
database.Execute("Update `Property` Set `Property`.`Value` = 'Test' WHERE `Property`.`Property` = 'ProductName'");
database.GenerateTransform(origDatabase, TRANSFORM);
database.CreateTransformSummaryInfo(origDatabase, TRANSFORM, TransformErrors.None, TransformValidations.None);
}
}
}
}
}

MongoDB GridFs with C#, how to store files such as images?

I'm developing a web app with mongodb as my back-end. I'd like to have users upload pictures to their profiles like a linked-in profile pic. I'm using an aspx page with MVC2 and I read that GridFs library is used to store large file types as binaries. I've looked everywhere for clues as how this is done, but mongodb doesn't have any documentation for C# api or GridFs C#. I'm baffled and confused, could really use another set of brains.
Anyone one know how to actually implement a file upload controller that stores an image uploaded by a user into a mongodb collection? Thanks a million!
I've tried variations of this to no avail.
Database db = mongo.getDB("Blog");
GridFile file = new GridFile(db);
file.Create("image.jpg");
var images = db.GetCollection("images");
images.Insert(file.ToDocument());

Following example show how to save file and read back from gridfs(using official mongodb driver):
var server = MongoServer.Create("mongodb://localhost:27020");
var database = server.GetDatabase("tesdb");
var fileName = "D:\\Untitled.png";
var newFileName = "D:\\new_Untitled.png";
using (var fs = new FileStream(fileName, FileMode.Open))
{
var gridFsInfo = database.GridFS.Upload(fs, fileName);
var fileId = gridFsInfo.Id;
ObjectId oid= new ObjectId(fileId);
var file = database.GridFS.FindOne(Query.EQ("_id", oid));
using (var stream = file.OpenRead())
{
var bytes = new byte[stream.Length];
stream.Read(bytes, 0, (int)stream.Length);
using(var newFs = new FileStream(newFileName, FileMode.Create))
{
newFs.Write(bytes, 0, bytes.Length);
}
}
}
Results:
File:
Chunks collection:
Hope this help.

The answers above are soon to be outdated now that the 2.1 RC-0 driver has been released.
The way to work with files in v2.1 MongoDB with GridFS can now be done this way:
using MongoDB.Bson;
using MongoDB.Driver;
using MongoDB.Driver.GridFS;
using System.IO;
using System.Threading.Tasks;
namespace MongoGridFSTest
{
class Program
{
static void Main(string[] args)
{
var client = new MongoClient("mongodb://localhost");
var database = client.GetDatabase("TestDB");
var fs = new GridFSBucket(database);
var id = UploadFile(fs);
DownloadFile(fs, id);
}
private static ObjectId UploadFile(GridFSBucket fs)
{
using (var s = File.OpenRead(#"c:\temp\test.txt"))
{
var t = Task.Run<ObjectId>(() => { return
fs.UploadFromStreamAsync("test.txt", s);
});
return t.Result;
}
}
private static void DownloadFile(GridFSBucket fs, ObjectId id)
{
//This works
var t = fs.DownloadAsBytesByNameAsync("test.txt");
Task.WaitAll(t);
var bytes = t.Result;
//This blows chunks (I think it's a driver bug, I'm using 2.1 RC-0)
var x = fs.DownloadAsBytesAsync(id);
Task.WaitAll(x);
}
}
}
This is taken from a diff on the C# driver tests here

This example will allow you to tie a document to an object
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using MongoDB.Driver;
using MongoDB.Driver.Linq;
using MongoDB.Bson;
using MongoDB.Driver.Builders;
using MongoDB.Driver.GridFS;
using System.IO;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
MongoServer ms = MongoServer.Create();
string _dbName = "docs";
MongoDatabase md = ms.GetDatabase(_dbName);
if (!md.CollectionExists(_dbName))
{
md.CreateCollection(_dbName);
}
MongoCollection<Doc> _documents = md.GetCollection<Doc>(_dbName);
_documents.RemoveAll();
//add file to GridFS
MongoGridFS gfs = new MongoGridFS(md);
MongoGridFSFileInfo gfsi = gfs.Upload(#"c:\mongodb.rtf");
_documents.Insert(new Doc()
{
DocId = gfsi.Id.AsObjectId,
DocName = #"c:\foo.rtf"
}
);
foreach (Doc item in _documents.FindAll())
{
ObjectId _documentid = new ObjectId(item.DocId.ToString());
MongoGridFSFileInfo _fileInfo = md.GridFS.FindOne(Query.EQ("_id", _documentid));
gfs.Download(item.DocName, _fileInfo);
Console.WriteLine("Downloaded {0}", item.DocName);
Console.WriteLine("DocName {0} dowloaded", item.DocName);
}
Console.ReadKey();
}
}
class Doc
{
public ObjectId Id { get; set; }
public string DocName { get; set; }
public ObjectId DocId { get; set; }
}