We Keep Coding

Cannot get link from file sent up to dropbox getting message WaitingForActivation

I'm attempting to upload a file and then get a link for it once it has been uploaded.
I seem to be sending up the file without a problem but when I go to obtain a link to the file, it sends back WaitingForActivation.
Here's my code:
private async Task ChunkUpload2(DropboxClient client, string folder, string fileName)
{
try
{
Console.WriteLine("Starting file upload...");
var path = $"{folder}/{Path.GetFileName(fileName)}";
// Chunk size is 128KB.
const int chunkSize = 4096 * 1024;
if (!File.Exists(fileName))
{
Console.WriteLine();
Console.WriteLine($"File does not exist: {fileName}");
Console.WriteLine();
return;
}
using (var stream = new MemoryStream(File.ReadAllBytes(fileName)))
{
int numChunks = (int) Math.Ceiling((double) stream.Length / chunkSize);
byte[] buffer = new byte[chunkSize];
string sessionId = null;
if (numChunks == 1)
{
using (var memStream = new MemoryStream(buffer, 0, chunkSize))
{
Console.WriteLine($"Sending file: {path}");
var tst = await client.Files.UploadAsync(path, WriteMode.Overwrite.Instance, body: memStream);
//Console.WriteLine(updated.Result);
}
}
else
{
for (var idx = 0; idx < numChunks; idx++)
{
Console.WriteLine($"Uploading chunk {idx + 1} / {numChunks}.");
var byteRead = stream.Read(buffer, 0, chunkSize);
using (MemoryStream memStream = new MemoryStream(buffer, 0, byteRead))
{
var cursor = new UploadSessionCursor(sessionId, (ulong) (chunkSize * idx));
if (idx == numChunks - 1)
{
Console.WriteLine($"Finalizing file: {path}");
var response = await client.Files.UploadSessionFinishAsync(cursor, new CommitInfo(path), memStream);
}
else
{
await client.Files.UploadSessionAppendV2Async(cursor, body: memStream);
}
}
}
}
var url = string.Empty;
var link = await client.Sharing.ListSharedLinksAsync(path);
if (link.Links.Count == 0)
{
var result = client.Sharing.CreateSharedLinkWithSettingsAsync(path);
url = result.Result.Url;
}
else
{
url = link.Links[0].Url;
}
Console.WriteLine();
Console.WriteLine("Dropbox Download Link:");
Console.WriteLine(url);
Console.WriteLine();
//Console.ReadKey();
}
}
catch (Exception e)
{
Console.WriteLine(e);
throw;
}
}
The last portion, at CreateSharedLinkWithSettingsAsync is where it's giving me back no share urls. Instead it returns the code above.
I'm getting this message:
Any suggestions on how to 'Active' this file? I'm assuming I'm missing something?

C# gRPC file streaming, original file smaller than the streamed one

I am having some problems with setting up a request-stream type gRPC architecture. The code below is just for testing purposes and it is missing various validation checks, but the main issue is that the original file is always smaller than the received one.
Could the cause here be encoding? It doesn't matter what the file type is, the end result is always that the file sizes are different.
Protobuf inteface:
syntax = "proto3";
package FileTransfer;
option csharp_namespace = "FileTransferProto";
service FileTransferService {
rpc DownloadFile(FileRequest) returns (stream ChunkMsg);
}
message ChunkMsg {
string FileName = 1;
int64 FileSize = 2;
bytes Chunk = 3;
}
message FileRequest {
string FilePath = 1;
}
Server side (sending):
public override async Task DownloadFile(FileRequest request, IServerStreamWriter<ChunkMsg> responseStream, ServerCallContext context)
{
string filePath = request.FilePath;
if (!File.Exists(filePath)) { return; }
FileInfo fileInfo = new FileInfo(filePath);
ChunkMsg chunk = new ChunkMsg();
chunk.FileName = Path.GetFileName(filePath);
chunk.FileSize = fileInfo.Length;
int fileChunkSize = 64 * 1024;
byte[] fileByteArray = File.ReadAllBytes(filePath);
byte[] fileChunk = new byte[fileChunkSize];
int fileOffset = 0;
while (fileOffset < fileByteArray.Length && !context.CancellationToken.IsCancellationRequested)
{
int length = Math.Min(fileChunkSize, fileByteArray.Length - fileOffset);
Buffer.BlockCopy(fileByteArray, fileOffset, fileChunk, 0, length);
fileOffset += length;
ByteString byteString = ByteString.CopyFrom(fileChunk);
chunk.Chunk = byteString;
await responseStream.WriteAsync(chunk).ConfigureAwait(false);
}
}
Client side (receiving):
public static async Task GetFile(string filePath)
{
var channel = Grpc.Net.Client.GrpcChannel.ForAddress("https://localhost:5001/", new GrpcChannelOptions
{
MaxReceiveMessageSize = 5 * 1024 * 1024, // 5 MB
MaxSendMessageSize = 5 * 1024 * 1024, // 5 MB
});
var client = new FileTransferProto.FileTransferService.FileTransferServiceClient(channel);
var request = new FileRequest { FilePath = filePath };
string tempFileName = $"temp_{DateTime.UtcNow.ToString("yyyyMMdd_HHmmss")}.tmp";
string finalFileName = tempFileName;
using (var call = client.DownloadFile(request))
{
await using (Stream fs = File.OpenWrite(tempFileName))
{
await foreach (ChunkMsg chunkMsg in call.ResponseStream.ReadAllAsync().ConfigureAwait(false))
{
Int64 totalSize = chunkMsg.FileSize;
string tempFinalFilePath = chunkMsg.FileName;
if (!string.IsNullOrEmpty(tempFinalFilePath))
{
finalFileName = chunkMsg.FileName;
}
fs.Write(chunkMsg.Chunk.ToByteArray());
}
}
}
if (finalFileName != tempFileName)
{
File.Move(tempFileName, finalFileName);
}
}

To add to Marc's answer, I feel like you can simplify your code a little bit.
using var fs = File.Open(filePath, System.IO.FileMode.Open);
int bytesRead;
var buffer = new byte[fileChunkSize];
while ((bytesRead = await fs.ReadAsync(buffer)) > 0)
{
await call.RequestStream.WriteAsync(new ChunkMsg
{
// Here the correct number of bytes must be sent which is starting from
// index 0 up to the number of read bytes from the file stream.
// If you solely pass 'buffer' here, the same bug would be present.
Chunk = ByteString.CopyFrom(buffer[0..bytesRead]),
});
}
I've used the array range operator from C# 8.0 which makes this cleaner or you can also use the overload of ByteString.CopyFrom which takes in an offset and count of how many bytes to include.

In your write loop, the chunk you actually send is for the oversized buffer, not accounting for length. This means that the last segment includes some garbage and is oversized. The received payload will be oversized by this same amount. So: make sure you account for length when constructing the chunk to send.

I tested the code and modified it to transfer the correct size.
The complete code is available at the following URL: https://github.com/lisa3907/grpc.fileTransfer
server-side-code
while (_offset < _file_bytes.Length)
{
if (context.CancellationToken.IsCancellationRequested)
break;
var _length = Math.Min(_chunk_size, _file_bytes.Length - _offset);
Buffer.BlockCopy(_file_bytes, _offset, _file_chunk, 0, _length);
_offset += _length;
_chunk.ChunkSize = _length;
_chunk.Chunk = ByteString.CopyFrom(_file_chunk);
await responseStream.WriteAsync(_chunk).ConfigureAwait(false);
}
client-side-code
await foreach (var _chunk in _call.ResponseStream.ReadAllAsync().ConfigureAwait(false))
{
var _total_size = _chunk.FileSize;
if (!String.IsNullOrEmpty(_chunk.FileName))
{
_final_file = _chunk.FileName;
}
if (_chunk.Chunk.Length == _chunk.ChunkSize)
_fs.Write(_chunk.Chunk.ToByteArray());
else
{
_fs.Write(_chunk.Chunk.ToByteArray(), 0, _chunk.ChunkSize);
Console.WriteLine($"final chunk size: {_chunk.ChunkSize}");
}
}

c# web request html response doesn't show well

I'm trying to retrieve a website using tcp and http requests, I added a textbox and a Go button , I type an address in the textbox and then I press the button to get to the website. it works well except it doesn't show me a complete page, for example when I try to reach www.google.com, it won't show google logo and it also keeps giving me warnings about js files.
here is the main part of my code , any help is deeply appreciated.
private async void button1_ClickAsync(object sender, EventArgs e)
{
string result = string.Empty;
using (var tcp = new TcpClient(textBox1.Text, 80))
using (var stream = tcp.GetStream())
{
tcp.SendTimeout = 500;
tcp.ReceiveTimeout = 1000;
var builder = new StringBuilder();
builder.AppendLine("GET /?scope=images&nr=1 HTTP/1.1");
builder.AppendLine("Host: " + textBox1.Text);
//builder.AppendLine("Content-Length: " + data.Length); // only for POST request
builder.AppendLine("Connection: close");
builder.AppendLine();
var header = Encoding.ASCII.GetBytes(builder.ToString());
await stream.WriteAsync(header, 0, header.Length);
//await stream.WriteAsync(data, 0, data.Length);
using (var memory = new MemoryStream())
{
await stream.CopyToAsync(memory);
memory.Position = 0;
var data = memory.ToArray();
var index = BinaryMatch(data, Encoding.ASCII.GetBytes("\r\n\r\n")) + 4;
var headers = Encoding.ASCII.GetString(data, 0, index);
memory.Position = index;
if (headers.IndexOf("Content-Encoding: gzip") > 0)
{
using (GZipStream decompressionStream = new GZipStream(memory, CompressionMode.Decompress))
using (var decompressedMemory = new MemoryStream())
{
decompressionStream.CopyTo(decompressedMemory);
decompressedMemory.Position = 0;
result = Encoding.UTF8.GetString(decompressedMemory.ToArray());
webBrowser2.DocumentText = result;
}
}
else
{
result = Encoding.UTF8.GetString(data, index, data.Length - index);
webBrowser2.DocumentText = result;
//result = Encoding.GetEncoding("gbk").GetString(data, index, data.Length - index);
}
}
//Debug.WriteLine(result);
//return result;
}
}

stream video in asp.net web api from database

i want to stream video from database through asp.net web api controller. i have done it from static file in my server(below code), but i can't accomplish the database mode.
here is my code (which i got from searching through web)
public class VideosController : ApiController
{
// GET api/values
public HttpResponseMessage Get(string filename)
{
var filePath = HttpContext.Current.Server.MapPath("~") + filename;
if (!File.Exists(filePath))
return new HttpResponseMessage(HttpStatusCode.NotFound);
var response = Request.CreateResponse();
response.Headers.AcceptRanges.Add("bytes");
var streamer = new FileStreamer();
streamer.FileInfo = new FileInfo(filePath);
response.Content = new PushStreamContent(streamer.WriteToStream, "video/mp4");
RangeHeaderValue rangeHeader = Request.Headers.Range;
if (rangeHeader != null)
{
long totalLength = streamer.FileInfo.Length;
var range = rangeHeader.Ranges.First();
streamer.Start = range.From ?? 0;
streamer.End = range.To ?? totalLength - 1;
response.Content.Headers.ContentLength = streamer.End - streamer.Start + 1;
response.Content.Headers.ContentRange = new ContentRangeHeaderValue(streamer.Start, streamer.End,
totalLength);
response.StatusCode = HttpStatusCode.PartialContent;
}
else
{
response.StatusCode = HttpStatusCode.OK;
}
return response;
}
class FileStreamer
{
public FileInfo FileInfo { get; set; }
public long Start { get; set; }
public long End { get; set; }
public async Task WriteToStream(Stream outputStream, HttpContent content, TransportContext context)
{
try
{
var buffer = new byte[65536];
using (var video = FileInfo.OpenRead())
{
if (End == -1)
{
End = video.Length;
}
var position = Start;
var bytesLeft = End - Start + 1;
video.Position = Start;
while (position <= End)
{
// what should i do here?
var bytesRead = video.Read(buffer, 0, (int)Math.Min(bytesLeft, buffer.Length));
await outputStream.WriteAsync(buffer, 0, bytesRead);
position += bytesRead;
bytesLeft = End - position + 1;
}
}
}
catch (Exception ex)
{
// fail silently
}
finally
{
outputStream.Close();
}
}
}
}
this is my HTML code:
<video width="640" height="480" controls="controls">
<source src="/api/Videos/?filename=sample.mp4" type="video/mp4">
</video>
there is a method ready for me (written by someone else) to get a range of file from engine (database) and its code is like this :
public byte[] Engine.DownloadStreamFile(Guid fileId, long from, long length)
i tried to read from this method and write on response output stream, but i couldn't. it seems i can't handle From and To receiving from google chrome. Any thoughts?

Based on the information you've provided, and assuming the method in your Engine class does what one would assume it does by name and signature, you should try replacing the file system stuff with your Engine.DownloadStreamFile method:
// what should i do here?
var bytesRead = video.Read(buffer, 0, (int)Math.Min(bytesLeft, buffer.Length));
// becomes
var bytesRead = Engine.DownloadStreamFile(this.fileId, this.Start, this.End);
You will obviously need to add a fileId field/property to your class instead of the FileInfo you have today.

asp.net core -"Multipart body length limit 16384 exceeded" error

I am working on a .net core project which has an web api to upload files. I have a middleware to check file's type, length etc.
I increased all limits on startup.cs file like this;
services.Configure<FormOptions>(f =>
{
f.MultipartBodyLengthLimit = Int32.MaxValue;
f.ValueLengthLimit = Int32.MaxValue;
f.BufferBodyLengthLimit = Int64.MaxValue;
f.MultipartBoundaryLengthLimit = Int32.MaxValue;
f.MultipartHeadersLengthLimit = Int32.MaxValue;
f.MemoryBufferThreshold = Int32.MaxValue;
f.ValueCountLimit = Int32.MaxValue;
});
But somehow error still shows up.
I even tried to create attribute class for limits, but no luck.
Here is the middleware class where i am checking the files:
if (authToken != String.Empty)
{
var ctype = context.Request.ContentType;
if (!IsMultipartContentType(context.Request.ContentType))
{
await context.Response.WriteAsync("Unexpected error.");
}
var boundary = GetBoundary(context.Request.ContentType);
var reader = new MultipartReader(boundary, streamBody);
reader.BodyLengthLimit = Int32.MaxValue;
var section = await reader.ReadNextSectionAsync();
while (section != null)
{
const int chunkSize = 256;
var buffer = new byte[chunkSize];
var fileName = GetFileName(section.ContentDisposition);
var bytesRead = 0;
using (var stream = new FileStream(fileName, FileMode.Append))
{
do
{
bytesRead = await section.Body.ReadAsync(buffer, 0, buffer.Length);
stream.Write(buffer, 0, bytesRead);
} while (bytesRead < 0);
}
if (!verifyUpcomingFiles(buffer))
{
await context.Response.WriteAsync("Undefined file type detected.");
}
else
{
//context.Request.Headers.Add("isVerified","verified");
await _next(context);
context.Request.ContentType = ctype;
await _next(context);
}
section = await reader.ReadNextSectionAsync();
}
}
else
{
await context.Response.WriteAsync("You are not allowed to complete this process.");
return;
}
I have been trying to make this work for 2 days.
I almost give up. Any help any help would be appreciated.