I created a simple Azure Function that creates thumbnails for images uploaded to a container on Azure which gets triggered by a BlobTrigger.
It's working fine but because my container has both image files as well as other types e.g. PDF, Excel, Word, etc., the function gets triggered by all these files.
I thought I could address this by making sure that we only process image files using the code below. It kind of works because it only processes image files but it still seems to create placeholder blobs for other file types in the target container.
For example, if it detects a file named myfile.pdf in the source container, it still creates a myfile.pdf in the target container but it's 0 Bytes.
How do I make sure that a non-image files completely get skipped and not even create placeholders in my target container?
[FunctionName("ImageResizer")]
public async Task Run([BlobTrigger("my-source-container/{name}", Connection = "myconnection")] Stream input, string name, [Blob("my-thumbnails-container/{name}", FileAccess.Write, Connection = "myconnection")] Stream outputBlob, ILogger log)
{
try
{
var fileExtension = FileUtils.GetFileExtension(name);
if (!string.IsNullOrEmpty(fileExtension))
{
if (fileExtension.ToLower() == "png" || fileExtension.ToLower() == "jpg" || fileExtension.ToLower() == "jpeg")
{
using (var image = Image.Load(input))
{
image.Mutate(x => x.Resize(new ResizeOptions
{
Size = new Size(150, 150),
Mode = ResizeMode.Crop
}));
using (var ms = new MemoryStream())
{
if(fileExtension.ToLower() == "png")
await image.SaveAsPngAsync(outputBlob);
else if(fileExtension.ToLower() == "jpg" || fileExtension.ToLower() == "jpeg")
await image.SaveAsJpegAsync(outputBlob);
}
}
log.LogInformation($"C# Blob trigger function Processed blob\n Name:{name} \n Size: {input.Length} Bytes");
}
}
}
catch (Exception ex)
{
log.LogInformation(ex.Message, null);
}
}
When using declarative bindings using attributes this is unavoidable unless you can somehow filter out unwanted blobs using the BlobTrigger attribute. It's default behavior for an output binding to expect the binding to be necessary so it is created as soon as the Function is executed.
However, with .Net languages you can use runtime bindings so only blobs that are actually handled will lead to output files, see the docs. That way you have more control regarding when an output blob is created.
[FunctionName("ImageResizer")]
public async Task Run([BlobTrigger("my-source-container/{name}", Connection = "myconnection")] Stream input, string name, IBinder binder, ILogger log)
{
try
{
var fileExtension = FileUtils.GetFileExtension(name);
if (!string.IsNullOrEmpty(fileExtension))
{
if (fileExtension.ToLower() == "png" || fileExtension.ToLower() == "jpg" || fileExtension.ToLower() == "jpeg")
{
using (var image = Image.Load(input))
{
image.Mutate(x => x.Resize(new ResizeOptions
{
Size = new Size(150, 150),
Mode = ResizeMode.Crop
}));
var attribute = new BlobAttribute("my-thumbnails-container/{name}", FileAccess.Write);
attribute.Connection = "myconnectionstring";
using (var ms = new MemoryStream())
using (var stream = await binder.BindAsync<Stream>(attribute))
{
if (fileExtension.ToLower() == "png")
await image.SaveAsPngAsync(stream);
else if (fileExtension.ToLower() == "jpg" || fileExtension.ToLower() == "jpeg")
await image.SaveAsJpegAsync(stream);
}
}
log.LogInformation($"C# Blob trigger function Processed blob\n Name:{name} \n Size: {input.Length} Bytes");
}
}
}
catch (Exception ex)
{
log.LogInformation(ex.Message, null);
}
}
Here is also a blogpost outlining what I just did.
Related
I have some code that uses Azure Storage as a repository for serialized JSON objects. I recently set up a method to go through and pick up every file named location.json containing a single object inside a number of directories.
When I first tested the code the deserialization failed because there were brackets around my object. However, when I downloaded the file, the array declaration was not there. I cannot for the life of me figure out where the array declaration is coming from in my code, and it's forcing me to change the way I think about using storage as a result. Has anyone else run into this?
Here's what I ended up using, which works but is also very slow.
public async Task<List<T>> GetAllAsync<T>(string delimiter)
{
var ret = new List<T>();
var blobs = client.GetBlobsByHierarchyAsync(delimiter:delimiter); // "*/location.json"
await foreach (var blobItem in blobs)
{
if (blobItem == null) continue;
log.LogInformation("{Prefix}",blobItem.Prefix);
var blob = client.GetBlobClient(blobItem.Blob.Name);
await using var stm = await blob.OpenReadAsync(new BlobOpenReadOptions(allowModifications: false)
{ Position = 0 });
if (stm == null) continue;
using var sr = new StreamReader(stm);
var s = await sr.ReadToEndAsync();
if (s.First() == '[')
{
log.LogInformation("{Type} serialized as array, taking 'first' item",typeof(T));
var list = JsonSerializer.Deserialize<List<T>>(s);
if (list == null)
{
log.LogInformation("deserialized a null array");
continue;
}
ret.Add(list[0]);
}
else
{
var l = JsonSerializer.Deserialize<T>(s);
ret.Add(l);
}
}
return ret;
}
So far, after some research from the internet I have been able to select .jpeg files from the computer and upload it to the firebase using Unity C#.
But, I cannot figure out, how I should modify the code below to use it for uploading .txt files as well.
If there is some other simpler way to do this task please tell that (if any). Otherwise, tell how I should modify this code so that it fulfills the purpose mentioned above.
using UnityEngine;
using System.Collections;
using System.Collections.Generic;
//For Picking files
using System.IO;
using SimpleFileBrowser;
//For firebase storage
using Firebase;
using Firebase.Extensions;
using Firebase.Storage;
public class UploadFile : MonoBehaviour
{
FirebaseStorage storage;
StorageReference storageReference;
// Start is called before the first frame update
void Start()
{
FileBrowser.SetFilters(true, new FileBrowser.Filter("Images", ".jpg", ".png"), new FileBrowser.Filter("Text Files", ".txt", ".pdf"));
FileBrowser.SetDefaultFilter(".jpg");
FileBrowser.SetExcludedExtensions(".lnk", ".tmp", ".zip", ".rar", ".exe");
storage = FirebaseStorage.DefaultInstance;
storageReference = storage.GetReferenceFromUrl("gs://app_name.appspot.com/");
}
public void OnButtonClick()
{
StartCoroutine(ShowLoadDialogCoroutine());
}
IEnumerator ShowLoadDialogCoroutine()
{
yield return FileBrowser.WaitForLoadDialog(FileBrowser.PickMode.FilesAndFolders, true, null, null, "Load Files and Folders", "Load");
Debug.Log(FileBrowser.Success);
if (FileBrowser.Success)
{
// Print paths of the selected files (FileBrowser.Result) (null, if FileBrowser.Success is false)
for (int i = 0; i < FileBrowser.Result.Length; i++)
Debug.Log(FileBrowser.Result[i]);
Debug.Log("File Selected");
byte[] bytes = FileBrowserHelpers.ReadBytesFromFile(FileBrowser.Result[0]);
//Editing Metadata
var newMetadata = new MetadataChange();
newMetadata.ContentType = "image/jpeg";
//Create a reference to where the file needs to be uploaded
StorageReference uploadRef = storageReference.Child("uploads/newFile.jpeg");
Debug.Log("File upload started");
uploadRef.PutBytesAsync(bytes, newMetadata).ContinueWithOnMainThread((task) => {
if (task.IsFaulted || task.IsCanceled)
{
Debug.Log(task.Exception.ToString());
}
else
{
Debug.Log("File Uploaded Successfully!");
}
});
}
}
}
As far as I understand it is basically already working as expected but you need to handle the different file types / extensions differently.
I think all you need to actually do is
use the correct file name + extension
use the correct ContentType depending on the file extension
So maybe something like
IEnumerator ShowLoadDialogCoroutine()
{
yield return FileBrowser.WaitForLoadDialog(FileBrowser.PickMode.FilesAndFolders, true, null, null, "Load Files and Folders", "Load");
Debug.Log(FileBrowser.Success);
if (!FileBrowser.Success)
{
yield break;
}
//foreach (var file in FileBrowser.Result)
//{
// Debug.Log(file);
//}
var file = FileBrowser.Result[0];
Debug.Log("File Selected: \"{file}\"");
// e.g. C:\someFolder/someFile.txt => someFile.txt
var fileNameWithExtension = file.Split('/', '\\').Last();
if (!fileNameWithExtension.Contains('.'))
{
throw new ArgumentException($"Selected file \"{file}\" is not a supported file!");
}
// e.g. someFile.txt => txt
var extensionWithoutDot = fileNameWithExtension.Split('.').Last();
// Get MIME type according to file extension
var contentType = extensionWithoutDot switch
{
"jpg" => $"image/jpeg",
"jpeg" => $"image/jpeg",
"png" => $"image/png",
"txt" => "text/plain",
"pdf" => "application/pdf",
_ => throw new ArgumentException($"Selected file \"{file}\" of type \"{extensionWithoutDot}\" is not supported!")
};
// Use dynamic content / MIME type
var newMetadata = new MetadataChange()
{
ContentType = contentType
};
// Use the actual selected file name including extension
StorageReference uploadRef = storageReference.Child($"uploads/{fileNameWithExtension}");
Debug.Log("File upload started");
uploadRef.PutBytesAsync(bytes, newMetadata).ContinueWithOnMainThread((task) =>
{
if (task.IsFaulted || task.IsCanceled)
{
Debug.LogException(task.Exception);
}
else
{
Debug.Log($"File \"{file}\" Uploaded Successfully!");
}
});
}
you most probably will want to replace the throw by proper error handling with user feedback later ;)
I have some data to save into a database.
I have created a web api post method to save data. Following is my post method:
[Route("PostRequirementTypeProcessing")]
public IEnumerable<NPAAddRequirementTypeProcessing> PostRequirementTypeProcessing(mdlAddAddRequirementTypeProcessing requTypeProcess)
{
mdlAddAddRequirementTypeProcessing rTyeProcessing = new mdlAddAddRequirementTypeProcessing();
rTyeProcessing.szDescription = requTypeProcess.szDescription;
rTyeProcessing.iRequirementTypeId = requTypeProcess.iRequirementTypeId;
rTyeProcessing.szRequirementNumber = requTypeProcess.szRequirementNumber;
rTyeProcessing.szRequirementIssuer = requTypeProcess.szRequirementIssuer;
rTyeProcessing.szOrganization = requTypeProcess.szOrganization;
rTyeProcessing.dIssuedate = requTypeProcess.dIssuedate;
rTyeProcessing.dExpirydate = requTypeProcess.dExpirydate;
rTyeProcessing.szSignedBy = requTypeProcess.szSignedBy;
rTyeProcessing.szAttachedDocumentNo = requTypeProcess.szAttachedDocumentNo;
if (String.IsNullOrEmpty(rTyeProcessing.szAttachedDocumentNo))
{
}
else
{
UploadFile();
}
rTyeProcessing.szSubject = requTypeProcess.szSubject;
rTyeProcessing.iApplicationDetailsId = requTypeProcess.iApplicationDetailsId;
rTyeProcessing.iEmpId = requTypeProcess.iEmpId;
NPAEntities context = new NPAEntities();
Log.Debug("PostRequirementTypeProcessing Request traced");
var newRTP = context.NPAAddRequirementTypeProcessing(requTypeProcess.szDescription, requTypeProcess.iRequirementTypeId,
requTypeProcess.szRequirementNumber, requTypeProcess.szRequirementIssuer, requTypeProcess.szOrganization,
requTypeProcess.dIssuedate, requTypeProcess.dExpirydate, requTypeProcess.szSignedBy,
requTypeProcess.szAttachedDocumentNo, requTypeProcess.szSubject, requTypeProcess.iApplicationDetailsId,
requTypeProcess.iEmpId);
return newRTP.ToList();
}
There is a field called 'szAttachedDocumentNo' which is a document that's being saved in the database as well.
After saving all data, I want the physical file of the 'szAttachedDocumentNo' to be saved on the server. So i created a method called "UploadFile" as follows:
[HttpPost]
public void UploadFile()
{
if (HttpContext.Current.Request.Files.AllKeys.Any())
{
// Get the uploaded file from the Files collection
var httpPostedFile = HttpContext.Current.Request.Files["UploadedFile"];
if (httpPostedFile != null)
{
// Validate the uploaded image(optional)
string folderPath = HttpContext.Current.Server.MapPath("~/UploadedFiles");
//string folderPath1 = Convert.ToString(ConfigurationManager.AppSettings["DocPath"]);
//Directory not exists then create new directory
if (!Directory.Exists(folderPath))
{
Directory.CreateDirectory(folderPath);
}
// Get the complete file path
var fileSavePath = Path.Combine(folderPath, httpPostedFile.FileName);
// Save the uploaded file to "UploadedFiles" folder
httpPostedFile.SaveAs(fileSavePath);
}
}
}
Before running the project, i debbugged the post method, so when it comes to "UploadFile" line, it takes me to its method.
From the file line, it skipped the remaining lines and went to the last line; what means it didn't see any file.
I am able to save everything to the database, just that i didn't see the physical file in the specified location.
Any help would be much appreciated.
Regards,
Somad
Makes sure the request "content-type": "multipart/form-data" is set
[HttpPost()]
public async Task<IHttpActionResult> UploadFile()
{
if (!Request.Content.IsMimeMultipartContent())
throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
try
{
MultipartMemoryStreamProvider provider = new MultipartMemoryStreamProvider();
await Request.Content.ReadAsMultipartAsync(provider);
if (provider.Contents != null && provider.Contents.Count == 0)
{
return BadRequest("No files provided.");
}
foreach (HttpContent file in provider.Contents)
{
string filename = file.Headers.ContentDisposition.FileName.Trim('\"');
byte[] buffer = await file.ReadAsByteArrayAsync();
using (MemoryStream stream = new MemoryStream(buffer))
{
// save the file whereever you want
}
}
return Ok("files Uploded");
}
catch (Exception ex)
{
return InternalServerError(ex);
}
}
Is there any equivalent of
Clipboard.GetImage().Save(FileName, Imaging.ImageFormat.Jpeg)
for UWP (Windows Universal Platform)?
I.e. saving the graphics image from clipboard into jpg format to file.
I am looking for example in vb.net/C#.
I have already started with
Dim datapackage = DataTransfer.Clipboard.GetContent()
If datapackage.Contains(StandardDataFormats.Bitmap) Then
Dim r As Windows.Storage.Streams.RandomAccessStreamReference = Await datapackage.GetBitmapAsync()
...
but I do not know how to continue (and even if I have even started correctly).
The first step is to try and get the image from the clipboard, if it exists:
var dataPackageView = Windows.ApplicationModel.DataTransfer.Clipboard.GetContent();
if (dataPackageView.Contains(StandardDataFormats.Bitmap))
{
IRandomAccessStreamReference imageReceived = null;
try
{
imageReceived = await dataPackageView.GetBitmapAsync();
}
catch (Exception ex)
{
}
If it exists, launch a file save picker, choose where to save the image, and copy the image stream to the new file.
if (imageReceived != null)
{
using (var imageStream = await imageReceived.OpenReadAsync())
{
var fileSave = new FileSavePicker();
fileSave.FileTypeChoices.Add("Image", new string[] { ".jpg" });
var storageFile = await fileSave.PickSaveFileAsync();
using (var stream = await storageFile.OpenAsync(FileAccessMode.ReadWrite))
{
await imageStream.AsStreamForRead().CopyToAsync(stream.AsStreamForWrite());
}
}
}
}
I want to enable drag and drop from our windows forms based application to Windows Explorer. The big problem: The files are stored in a database, so I need to use delayed data rendering. There is an article on codeproject.com, but the author is using a H_GLOBAL object which leads to memory problems with files bigger than aprox. 20 MB. I haven't found a working solution for using an IStream Object instead. I think this must be possible to implement, because this isn't an unusual case. (A FTP program needs such a feature too, for example)
Edit: Is it possible to get an event when the user drops the file? So I could for example copy it to temp and the explorer gets it from there? Maybe there is an alternative approach for my problem...
AFAIK, there is not working article about this for .net. So you should write it by yourself, this is somewhat complicate, because .net DataObject class is limited. I have working example of the opposite task (accepting delayed rendering files from explorer), but it is easier, because I do not needed own IDataObject implementation.
So your task will be:
Find working IDataObject implementation in .net. I recommend you look here (Shell Style Drag and Drop in .NET (WPF and WinForms))
You also need an IStream wrapper for managed stream (it is relatively easy to implement)
Implement delayed rendering using information from MSDN (Shell Clipboard Formats)
This is the starting point, and in general enough information to implement such feature. With bit of patience and several unsuccessful attempts you will do it :)
Update: The following code lacks many necessary methods and functions, but the main logic is here.
// ...
private static IEnumerable<IVirtualItem> GetDataObjectContent(System.Windows.Forms.IDataObject dataObject)
{
if (dataObject == null)
return null;
List<IVirtualItem> Result = new List<IVirtualItem>();
bool WideDescriptor = dataObject.GetDataPresent(ShlObj.CFSTR_FILEDESCRIPTORW);
bool AnsiDescriptor = dataObject.GetDataPresent(ShlObj.CFSTR_FILEDESCRIPTORA);
if (WideDescriptor || AnsiDescriptor)
{
IDataObject NativeDataObject = dataObject as IDataObject;
if (NativeDataObject != null)
{
object Data = null;
if (WideDescriptor)
Data = dataObject.GetData(ShlObj.CFSTR_FILEDESCRIPTORW);
else
if (AnsiDescriptor)
Data = dataObject.GetData(ShlObj.CFSTR_FILEDESCRIPTORA);
Stream DataStream = Data as Stream;
if (DataStream != null)
{
Dictionary<string, VirtualClipboardFolder> FolderMap =
new Dictionary<string, VirtualClipboardFolder>(StringComparer.OrdinalIgnoreCase);
BinaryReader Reader = new BinaryReader(DataStream);
int Count = Reader.ReadInt32();
for (int I = 0; I < Count; I++)
{
VirtualClipboardItem ClipboardItem;
if (WideDescriptor)
{
FILEDESCRIPTORW Descriptor = ByteArrayHelper.ReadStructureFromStream<FILEDESCRIPTORW>(DataStream);
if (((Descriptor.dwFlags & FD.FD_ATTRIBUTES) > 0) && ((Descriptor.dwFileAttributes & FileAttributes.Directory) > 0))
ClipboardItem = new VirtualClipboardFolder(Descriptor);
else
ClipboardItem = new VirtualClipboardFile(Descriptor, NativeDataObject, I);
}
else
{
FILEDESCRIPTORA Descriptor = ByteArrayHelper.ReadStructureFromStream<FILEDESCRIPTORA>(DataStream);
if (((Descriptor.dwFlags & FD.FD_ATTRIBUTES) > 0) && ((Descriptor.dwFileAttributes & FileAttributes.Directory) > 0))
ClipboardItem = new VirtualClipboardFolder(Descriptor);
else
ClipboardItem = new VirtualClipboardFile(Descriptor, NativeDataObject, I);
}
string ParentFolder = Path.GetDirectoryName(ClipboardItem.FullName);
if (string.IsNullOrEmpty(ParentFolder))
Result.Add(ClipboardItem);
else
{
VirtualClipboardFolder Parent = FolderMap[ParentFolder];
ClipboardItem.Parent = Parent;
Parent.Content.Add(ClipboardItem);
}
VirtualClipboardFolder ClipboardFolder = ClipboardItem as VirtualClipboardFolder;
if (ClipboardFolder != null)
FolderMap.Add(PathHelper.ExcludeTrailingDirectorySeparator(ClipboardItem.FullName), ClipboardFolder);
}
}
}
}
return Result.Count > 0 ? Result : null;
}
// ...
public VirtualClipboardFile : VirtualClipboardItem, IVirtualFile
{
// ...
public Stream Open(FileMode mode, FileAccess access, FileShare share, FileOptions options, long startOffset)
{
if ((mode != FileMode.Open) || (access != FileAccess.Read))
throw new ArgumentException("Only open file mode and read file access supported.");
System.Windows.Forms.DataFormats.Format Format = System.Windows.Forms.DataFormats.GetFormat(ShlObj.CFSTR_FILECONTENTS);
if (Format == null)
return null;
FORMATETC FormatEtc = new FORMATETC();
FormatEtc.cfFormat = (short)Format.Id;
FormatEtc.dwAspect = DVASPECT.DVASPECT_CONTENT;
FormatEtc.lindex = FIndex;
FormatEtc.tymed = TYMED.TYMED_ISTREAM | TYMED.TYMED_HGLOBAL;
STGMEDIUM Medium;
FDataObject.GetData(ref FormatEtc, out Medium);
try
{
switch (Medium.tymed)
{
case TYMED.TYMED_ISTREAM:
IStream MediumStream = (IStream)Marshal.GetTypedObjectForIUnknown(Medium.unionmember, typeof(IStream));
ComStreamWrapper StreamWrapper = new ComStreamWrapper(MediumStream, FileAccess.Read, ComRelease.None);
// Seek from beginning
if (startOffset > 0)
if (StreamWrapper.CanSeek)
StreamWrapper.Seek(startOffset, SeekOrigin.Begin);
else
{
byte[] Null = new byte[256];
int Readed = 1;
while ((startOffset > 0) && (Readed > 0))
{
Readed = StreamWrapper.Read(Null, 0, (int)Math.Min(Null.Length, startOffset));
startOffset -= Readed;
}
}
StreamWrapper.Closed += delegate(object sender, EventArgs e)
{
ActiveX.ReleaseStgMedium(ref Medium);
Marshal.FinalReleaseComObject(MediumStream);
};
return StreamWrapper;
case TYMED.TYMED_HGLOBAL:
byte[] FileContent;
IntPtr MediumLock = Windows.GlobalLock(Medium.unionmember);
try
{
long Size = FSize.HasValue ? FSize.Value : Windows.GlobalSize(MediumLock).ToInt64();
FileContent = new byte[Size];
Marshal.Copy(MediumLock, FileContent, 0, (int)Size);
}
finally
{
Windows.GlobalUnlock(Medium.unionmember);
}
ActiveX.ReleaseStgMedium(ref Medium);
Stream ContentStream = new MemoryStream(FileContent, false);
ContentStream.Seek(startOffset, SeekOrigin.Begin);
return ContentStream;
default:
throw new ApplicationException(string.Format("Unsupported STGMEDIUM.tymed ({0})", Medium.tymed));
}
}
catch
{
ActiveX.ReleaseStgMedium(ref Medium);
throw;
}
}
// ...
Googlers may find this useful: download a file using windows IStream