Correct usage of ReadOnlySequence<T> on implementing ReadOnlySequenceSegment<T> - c#

I'm trying to implement a network library to complete my from scratch library Astron since a few days and I have a few questions. In order to handle Nagle correctly, I must have a frame parsing logic. So while doing that using the new pipeline API I noticed that output was provided as a read-only sequence, therefore I had to make a custom SequenceReader but I'm having some problems while testing it. I think I've misunderstood the new Span<T> and Memory<T> API :/
Is it relevant to mock the sequence segment?
In order to test as many behaviors as possible, I had to implement the ReadOnlySequenceSegment<T> abstract class, which I did using the Moq library to know when the next segment has been getting, here it is :
public class SequenceSegment : ReadOnlySequenceSegment<byte>
{
private readonly Mock<SequenceSegment> _mock;
private ReadOnlySequenceSegment<byte> _next;
public new ReadOnlySequenceSegment<byte> Next
{
get
{
var mockNext = _mock.Object.Next; // simulate get from the mock
return _next;
}
protected set => _next = value;
}
public SequenceSegment(ReadOnlyMemory<byte> memory)
{
Memory = memory;
_mock = new Mock<SequenceSegment>(memory);
}
public SequenceSegment Add(ReadOnlyMemory<byte> mem)
{
var segment = new SequenceSegment(mem)
{
RunningIndex = RunningIndex + Memory.Length
};
Next = segment;
return segment;
}
public void VerifyGetNext() => _mock.VerifyGet(ss => ss.Next);
}
As you can see I had to override the Next property with the new keyword which is a bad practice but I assume while testing it's fine?
Here is the test that doesn't pass :
[Fact]
public void TryRead_ShouldReturnTrue_OnSegmentSplitted()
{
var buffer = _createBuffer(); // int + int = 8bytes buffer
var firstSegment = new SequenceSegment(buffer.Slice(0, 3));
var secondSegment = firstSegment.Add(buffer.Slice(3, 5));
var input = new ReadOnlySequence<byte>(
firstSegment, 0,
secondSegment, 4);
var reader = new SequenceReader(input);
reader.TryRead(BinaryPrimitives.ReadInt32LittleEndian, out var firstValue); // throw here
reader.TryRead(BinaryPrimitives.ReadInt32LittleEndian, out var secondValue);
Assert.Equal(_firstValue, firstValue);
Assert.Equal(_secondValue, secondValue);
firstSegment.VerifyGetNext();
}
Test output :
Message: System.ArgumentOutOfRangeException : Specified argument was out of the range of valid values.
Parameter name: start
I commented the line that throws the exception on the test, so I assume my sequence logic is okay? Let's see the code of my SequenceReader with the line that throws commented :
public class SequenceReader
{
private const int _maxStackalloc = 128;
protected ReadOnlySequence<byte> _input;
public SequenceReader(ReadOnlySequence<byte> input) => _input = input;
public delegate T ReadDelegate<out T>(ReadOnlySpan<byte> src);
/// <summary>
/// Try to read a <see cref="T"/> with the <see cref="ReadDelegate{T}"/> specified as an arg.
/// The <see cref="SequenceReader"/> then advance the current position according to the size of <see cref="T"/>.
/// <see cref="T"/> must be a struct :
/// <see cref="byte"/>, <see cref="sbyte"/>, <see cref="bool"/>, <see cref="short"/>,
/// <see cref="ushort"/>, <see cref="int"/>, <see cref="uint"/>, <see cref="long"/>,
/// <see cref="ulong"/>, <see cref="float"/>, <see cref="double"/>, <see cref="decimal"/>,
/// </summary>
/// <typeparam name="T">The type to read.</typeparam>
/// <param name="read">The delegate to read the <see cref="T"/>. Must be a method from <see cref="BinaryPrimitives"/></param>
/// <param name="result">The result returned.</param>
/// <returns>Returns true if the read was successful, else returns false.</returns>
public unsafe bool TryRead<T>(ReadDelegate<T> read, out T result) where T : unmanaged
{
result = default;
var size = sizeof(T);
if (size > _maxStackalloc) return false;
if (size > _input.Length) return false;
if (_input.First.Length >= size)
result = read(_input.First.Span);
else
{
Span<byte> local = stackalloc byte[size];
_input.Slice(size).CopyTo(local); // throws at the slice
result = read(local);
}
_input = _input.Slice(size);
return true;
}
}
I already tried to change the line to _input.Slice(0, size) but nothing changed, also not on this successful test :
[Fact]
public void TryRead_ShouldReturnTrue_OnSegmentComplete()
{
var input = new ReadOnlySequence<byte>(_createBuffer());
var reader = new SequenceReader(input);
reader.TryRead(BinaryPrimitives.ReadInt32LittleEndian, out var firstValue);
reader.TryRead(BinaryPrimitives.ReadInt32LittleEndian, out var secondValue);
Assert.Equal(_firstValue, firstValue);
Assert.Equal(_secondValue, secondValue);
}
I'm wondering what I'm doing wrong, please if you have any idea it'd be awesome if you can help me!

Related

C# - Odd Null Reference Exception during testing, why does this happen?

This references my last question which appears to have been abandoned. I am experiencing an odd "bug" if you will with C# and MS VS 2015. To reproduce the error, follow the steps:
Open console app project and copy paste code below.
Set a break point here:
First run code past break point, it works! :D
Then run code again but this time STOP at the break point and DRAG the executing statement cursor INTO the if statement from here:
to here:
Hit Continue and an NRE exception is thrown. Why does this happen? Is it just me? What is the technical explination for this?
CODE:
using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace testapp
{
class Program
{
static void Main(string[] args)
{
FILECollection randomCollection = new FILECollection();
// Fill with junk test data:
for(int i = 0; i<10; i++)
{
FILE junkfile = new FILE() { fileName = i.ToString(), folderName = i.ToString(), fileHashDigest = new byte[1] };
randomCollection.Add(junkfile);
}
if (true)
{
Console.WriteLine("testing this weird exception issue...");
FILE test;
test = new FILE();
test.fileName = "3";
test.folderName = "3";
test.fileHashDigest = new byte[1];
FILE exists = randomCollection.Where(f => f.fileName == test.fileName &&
f.fileHashDigest.SequenceEqual(test.fileHashDigest)).First();
}
}
}
public class FILE
{
public FILE() { _fileName = "";}
private string _fileName;
public string fileName
{
get
{
if (false)
return this._fileName.ToUpper();
else
return this._fileName;
}
set
{
if (false)
this._fileName = value.ToUpper();
else
this._fileName = value;
}
}
public string folderName { get; set; }
public byte[] fileHashDigest { get; set; }
}
public class FILECollection : IEnumerable<FILE>, ICollection<FILE>
{
private HashSet<FILE> svgHash;
private static List<FILE> PreallocationList;
public string FileName = "N/A";
/// <summary>
/// Default Constructor, will not
/// preallocate memory.
/// </summary>
/// <param name="PreallocationSize"></param>
public FILECollection()
{
this.svgHash = new HashSet<FILE>();
this.svgHash.Clear();
}
/// <summary>
/// Overload Constructor Preallocates
/// memory to be used for the new
/// FILE Collection.
/// </summary>
public FILECollection(int PreallocationSize, string fileName = "N/A", int fileHashDigestSize = 32)
{
FileName = fileName;
PreallocationList = new List<FILE>(PreallocationSize);
for (int i = 0; i <= PreallocationSize; i++)
{
byte[] buffer = new byte[fileHashDigestSize];
FILE preallocationSVG = new FILE()
{
fileName = "",
folderName = "",
fileHashDigest = buffer
};
PreallocationList.Add(preallocationSVG);
}
this.svgHash = new HashSet<FILE>(PreallocationList);
this.svgHash.Clear(); // Capacity remains unchanged until a call to TrimExcess is made.
}
/// <summary>
/// Add an FILE file to
/// the FILE Collection.
/// </summary>
/// <param name="svg"></param>
public void Add(FILE svg)
{
this.svgHash.Add(svg);
}
/// <summary>
/// Removes all elements
/// from the FILE Collection
/// </summary>
public void Clear()
{
svgHash.Clear();
}
/// <summary>
/// Determine if the FILE collection
/// contains the EXACT FILE file, folder,
/// and byte[] sequence. This guarantees
/// that the collection contains the EXACT
/// file you are looking for.
/// </summary>
/// <param name="item"></param>
/// <returns></returns>
public bool Contains(FILE item)
{
return svgHash.Any(f => f.fileHashDigest.SequenceEqual(item.fileHashDigest) &&
f.fileName == item.fileName &&
f.folderName == item.folderName);
}
/// <summary>
/// Determine if the FILE collection
/// contains the same file and folder name,
/// byte[] sequence is not compared. The file and folder
/// name may be the same but this does not guarantee the
/// file contents are exactly the same. Use Contains() instead.
/// </summary>
/// <param name="item"></param>
/// <returns></returns>
public bool ContainsPartially(FILE item)
{
return svgHash.Any(f => f.fileName == item.fileName &&
f.folderName == item.folderName);
}
/// <summary>
/// Returns the total number
/// of FILE files in the Collection.
/// </summary>
public int Count
{ get { return svgHash.Count(); } }
public bool IsReadOnly
{ get { return true; } }
public void CopyTo(FILE[] array, int arrayIndex)
{
svgHash.CopyTo(array, arrayIndex);
}
public bool Remove(FILE item)
{
return svgHash.Remove(item);
}
public IEnumerator<FILE> GetEnumerator()
{
return svgHash.GetEnumerator();
}
IEnumerator IEnumerable.GetEnumerator()
{
return svgHash.GetEnumerator();
}
}
}
I think either I am debugging in a terribly wrong way, or Microsoft should take a look at this. It's like future code is breaking current code...which is impossible!
OK here's my best guess..
First, as I mentioned in the comments, the exception doesn't occur if you comment out the line FILE exists = randomCollection.Where(f => f.fileName == test.fileName && f.fileHashDigest.SequenceEqual(test.fileHashDigest)).First()‌​;
Second, I noticed the same behavior can be reproduced with the following code:
if (true)
{
object o;
o = new object();
Func<bool> m = () => o == null;
}
i.e. the cause seems to be related to the variable being used in a lambda expression. So, looking at the same code snippet above in ILSpy I get the following:
Program.<>c__DisplayClass0_0 <>c__DisplayClass0_ = new Program.<>c__DisplayClass0_0();
<>c__DisplayClass0_.o = new object();
Func<bool> func = new Func<bool>(<>c__DisplayClass0_.<Main>b__0);
so my best guess is that the NullReferenceException refers to <>c__DisplayClass0_ intance being null - and I'm therefore inclined to believe that the stepping through the if(true) actually skipped the first line where <>c__DisplayClass0_ is instantiated

Handle multiple binding errors from ModelBindingException in NancyFX when binding to JSON in Request.Body

I have a post route that accepts some JSON payload in the request body.
Post["/myroute/}"] = _ =>
{
try
{
var model = this.Bind<MyModel>();
}
catch (ModelBindingException e)
{
//PropertyBindException list is empty here,
//so only the first exception can be handled...
}
}
If there are multiple invalid data types (i.e. if there are several int properties defined in MyModel, and a user posts strings for those properties), I would like pass back a nice list of these errors, similar to how would use ModelState dictionary in a vanilla ASP.NET MVC application.
How can I accomplish this type of exception handling when attempting to bind the JSON payload in the request body to my Model in NancyFX?
Update:
Looking through the DefaultBinder in the Nancy source here:
https://github.com/sloncho/Nancy/blob/master/src/Nancy/ModelBinding/DefaultBinder.cs
The problem I see is that in this block:
try
{
var bodyDeserializedModel = this.DeserializeRequestBody(bindingContext);
if (bodyDeserializedModel != null)
{
UpdateModelWithDeserializedModel(bodyDeserializedModel, bindingContext);
}
}
catch (Exception exception)
{
if (!bindingContext.Configuration.IgnoreErrors)
{
throw new ModelBindingException(modelType, innerException: exception);
}
}
The Deserialize call seems to be "all or nothing" and it is handled by a plain Exception, not a ModelBindException, so I cannot see any PropertyBindExceptions here either.
Should I be needing to implement something custom for this...?
Add your own custom body serializer that uses newtonsoft to ignore errors:
public class CustomBodyDeserializer : IBodyDeserializer
{
private readonly MethodInfo deserializeMethod = typeof(JavaScriptSerializer).GetMethod("Deserialize", BindingFlags.Instance | BindingFlags.Public);
private readonly JsonConfiguration jsonConfiguration;
private readonly GlobalizationConfiguration globalizationConfiguration;
/// <summary>
/// Initializes a new instance of the <see cref="JsonBodyDeserializer"/>,
/// with the provided <paramref name="environment"/>.
/// </summary>
/// <param name="environment">An <see cref="INancyEnvironment"/> instance.</param>
public CustomBodyDeserializer(INancyEnvironment environment)
{
this.jsonConfiguration = environment.GetValue<JsonConfiguration>();
this.globalizationConfiguration = environment.GetValue<GlobalizationConfiguration>();
}
/// <summary>
/// Whether the deserializer can deserialize the content type
/// </summary>
/// <param name="mediaRange">Content type to deserialize</param>
/// <param name="context">Current <see cref="BindingContext"/>.</param>
/// <returns>True if supported, false otherwise</returns>
public bool CanDeserialize(MediaRange mediaRange, BindingContext context)
{
return Json.IsJsonContentType(mediaRange);
}
/// <summary>
/// Deserialize the request body to a model
/// </summary>
/// <param name="mediaRange">Content type to deserialize</param>
/// <param name="bodyStream">Request body stream</param>
/// <param name="context">Current context</param>
/// <returns>Model instance</returns>
public object Deserialize(MediaRange mediaRange, Stream bodyStream, BindingContext context)
{
//var serializer = new JavaScriptSerializer(this.jsonConfiguration, this.globalizationConfiguration);
//serializer.RegisterConverters(this.jsonConfiguration.Converters, this.jsonConfiguration.PrimitiveConverters);
if (bodyStream.CanSeek)
{
bodyStream.Position = 0;
}
string bodyText;
using (var bodyReader = new StreamReader(bodyStream))
{
bodyText = bodyReader.ReadToEnd();
}
// var genericDeserializeMethod = this.deserializeMethod.MakeGenericMethod(context.DestinationType);
// var deserializedObject = genericDeserializeMethod.Invoke(serializer, new object[] { bodyText });
object deserializedObject = JsonConvert.DeserializeObject(bodyText, context.DestinationType, new JsonSerializerSettings
{
Error = HandleDeserializationError
});
return deserializedObject;
}
public void HandleDeserializationError(object sender, Newtonsoft.Json.Serialization.ErrorEventArgs errorArgs)
{
string currentError = errorArgs.ErrorContext.Error.Message;
errorArgs.ErrorContext.Handled = true;
}
}

Redefine massive of fixed size

I have an array size of 10. it must contains ten last values of incoming parameters (number of incoming parameters is nearly 3k) I have some logic in following loop:
for (int i=0; i<incomingLength; i++)
{
//and here I also need to rewrite this array size of 10 with new incomingValue(i)
}
perhaps it is primitive but I am stuck((
You can use a "Circular Buffer" for this.
Here's a sample implementation (parameter validation omitted for brevity):
using System;
using System.Collections;
using System.Collections.Generic;
using System.Diagnostics;
namespace Demo
{
public class CircularBuffer<T>: IEnumerable<T>
{
/// <summary>Constructor.</summary>
/// <param name="capacity">The maximum capacity of the buffer.</param>
public CircularBuffer(int capacity)
{
// The reason for this +1 is to simplify the logic - we can use "front == back" to indicate an empty buffer.
_buffer = new T[capacity+1];
}
/// <summary>The buffer capacity.</summary>
public int Capacity
{
get
{
return _buffer.Length - 1;
}
}
/// <summary>The number of elements currently stored in the buffer.</summary>
public int Count
{
get
{
int result = _back - _front;
if (result < 0)
result += _buffer.Length;
return result;
}
}
/// <summary>Is the buffer empty?</summary>
public bool IsEmpty
{
get
{
return this.Count == 0;
}
}
/// <summary>Is the buffer full? (i.e. has it reached its capacity?)</summary>
public bool IsFull
{
get
{
return nextSlot(_back) == _front;
}
}
/// <summary>Empties the buffer.</summary>
public void Empty()
{
_front = _back = 0;
Array.Clear(_buffer, 0, _buffer.Length); // Destroy any old references so they can be GCed.
}
/// <summary>Add an element to the buffer, overwriting the oldest element if the buffer is full.</summary>
/// <param name="newItem">The element to add.</param>
public void Add(T newItem)
{
_buffer[_back] = newItem;
_back = nextSlot(_back);
if (_back == _front) // Buffer is full?
{
_front = nextSlot(_front); // Bump the front, overwriting the current front.
_buffer[_back] = default(T); // Remove the old front value.
}
}
/// <summary>
/// The typesafe enumerator. Elements are returned in oldest to newest order.
/// This is not threadsafe, so if you are enumerating the buffer while another thread is changing it you will run
/// into threading problems. Therefore you must use your own locking scheme to avoid the problem.
/// </summary>
public IEnumerator<T> GetEnumerator()
{
for (int i = _front; i != _back; i = nextSlot(i))
yield return _buffer[i];
}
/// <summary>The non-typesafe enumerator.</summary>
IEnumerator IEnumerable.GetEnumerator()
{
return GetEnumerator(); // Implement in terms of the typesafe enumerator.
}
/// <summary>Calculates the index of the slot following the specified one, wrapping if necessary.</summary>
private int nextSlot(int slot)
{
return (slot + 1) % _buffer.Length;
}
/// <summary>
/// The index of the element at the front of the buffer.
/// If this equals _back, the buffer is empty.
/// </summary>
private int _front;
/// <summary>
/// The index of the first element BEYOND the last used element of the buffer.
/// Therefore this indicates where the next added element will go.
/// </summary>
private int _back;
/// <summary>The underlying buffer. This has a length one greater than the actual capacity.</summary>
private readonly T[] _buffer;
}
internal class Program
{
private void run()
{
CircularBuffer<int> buffer = new CircularBuffer<int>(10);
for (int i = 0; i < 20; ++i)
buffer.Add(i);
foreach (int n in buffer)
Console.WriteLine(n); // Prints 10..19
}
private static void Main()
{
new Program().run();
}
}
}
use array.copy
var arr1 = new int[] { 1, 2, 3};
var arr2 = new int[] { 4, 5};
var target = new int[arr1.Length + arr2.Length];
Array.Copy(arr1, target, arr1.Length);
Array.Copy(arr2, 0, target, arr1.Length, arr2.Length);
this will combine two arrays. you can modify indexes as you like

The best overloaded method match for has some invalid arguments

I am using VS 2010 C#. My code is as follows:
using System;
using System.Collections.Generic;
using System.Text;
using System.Threading;
using MarketplaceWebServiceOrders;
using MarketplaceWebServiceOrders.Model;
namespace FetchNewOrdersJob
{
public class MarketplaceWebServiceOrders
{
private volatile bool isRunning;
private OrderFetcher orderFetcher;
private TimeSpan _checkOrdersInterval = TimeSpan.FromMinutes(15.0);
/// <summary>
/// Gets or sets the order check interval. Defaults to 15 minutes.
/// </summary>
public TimeSpan CheckOrdersInterval
{
get { return _checkOrdersInterval; }
set { _checkOrdersInterval = value; }
}
/// <summary>
/// Internal method to handle an order.
/// </summary>
protected virtual void HandleOrder(Order order)
{
Console.WriteLine("Processing Order:");
Console.WriteLine("---------------------------------------------------");
Console.WriteLine(order.ToString());
// Fetch the order items in each order
orderFetcher.FetchOrderItems(order.AmazonOrderId, delegate(OrderItem item)
{
Console.WriteLine("\tProcessing Order Item");
Console.WriteLine("\t---------------------------------------------------"); // Process order item here.
Console.WriteLine("\t" + item.ToString().Replace("\n", "\n\t"));
});
Console.WriteLine("=================================================");
Console.WriteLine();
}
/// <summary>
/// Method to continuously check orders over an interval, and list OrderItems for those Orders.
/// </summary>
private void OrdersJobThread(object obj)
{
orderFetcher.ProcessOrder += HandleOrder;
if (this.CheckOrdersInterval == TimeSpan.MinValue)
{
throw new ArgumentException("The CheckOrdersInterval TimeSpan cannot be zero.", "CheckOrdersInterval");
}
DateTime startCheckInterval = DateTime.Now.Subtract(CheckOrdersInterval);
// Continue forever until the isRunning flag is cleared.
while (isRunning)
{
try
{
// Check the orders for this interval.
DateTime checkInterval = startCheckInterval;
startCheckInterval = DateTime.Now.Subtract(TimeSpan.FromMinutes(3.0));
Console.WriteLine("Fetching orders from " + checkInterval.ToString() + " to " + startCheckInterval.ToString());
orderFetcher.FetchOrders(checkInterval, startCheckInterval);
// Wait for the next interval.
Console.WriteLine("Fetch complete. Sleeping until next interval.");
while (isRunning && DateTime.Now.Subtract(startCheckInterval) < CheckOrdersInterval)
{
Thread.Sleep(1000);
}
}
catch(Exception err)
{
Console.WriteLine("Error: " + err.Message + ". Orders job thread is exiting.");
isRunning = false;
}
}
}
/// <summary>
/// Sample code to invoke the OrderFetcher.
/// </summary>
/// <param name="service">MarketplaceWebServiceOrders object.</param>
/// <param name="sellerId">The seller Id.</param>
/// <param name="marketplaceIdList">List of marketplaces passed in to the GetOrders call.</param>
public static void InvokeOrderFetcherSample(
MarketplaceWebServiceOrders service,
string sellerId,
string [] marketplaceIdList
)
{
// Create a FetchOrderUpdates job with the default time span.
MarketplaceWebServiceOrders job = new MarketplaceWebServiceOrders();
job.isRunning = true;
job.orderFetcher = new OrderFetcher(service, sellerId, marketplaceIdList);
Thread jobThread = new Thread(job.OrdersJobThread);
jobThread.IsBackground = true;
jobThread.Start();
// Pause on the main thread for one hour or until the thread exits, then end the job.
jobThread.Join(1000 * 60 * 60);
job.isRunning = false;
// Block until the thread terminates to prevent any requests in progress from being aborted.
while (jobThread.IsAlive)
{
Thread.Sleep(1000);
}
}
}
}
I get two errors and can't seem to figure this out
Argument 1: cannot convert from 'FetchNewOrdersJob.MarketplaceWebServiceOrders' to 'MarketplaceWebServiceOrders.MarketplaceWebServiceOrders'
The best overloaded method match for 'MarketplaceWebServiceOrders.OrderFetcher.OrderFetcher(MarketplaceWebServiceOrders.MarketplaceWebServiceOrders, string, string[])' has some invalid arguments
The Line number of the error is 112: job.orderFetcher = new OrderFetcher(service, sellerId, marketplaceIdList);
OrderFetcher.cs
using System;
using System.Collections.Generic;
using System.Text;
using MarketplaceWebServiceOrders.Model;
namespace MarketplaceWebServiceOrders
{
/// <summary>
/// Sample helper class to Fetch Orders and OrderItems using the Amazon MWS Orders API.
/// </summary>
public class OrderFetcher
{
public delegate void RetriableMethodCall();
public delegate void ProcessOrderHandler(Order order);
public delegate void ProcessOrderItemHandler(OrderItem orderItem);
/// <summary>
/// Default amount of time, in milliseconds, to sleep if a request is throttled; default to 1 per 10 minutes.
/// </summary>
public const int DEFAULT_THROTTLED_WAIT_TIMEOUT = 10 * 60 * 1000;
/// <summary>
/// Default throttling limit for ListOrders calls; default to 1 per 12 seconds.
/// </summary>
private const int LIST_ORDERS_DEFAULT_THROTTLE_LIMIT = 12 * 1000;
/// <summary>
/// Default throttling limit for ListOrderItems calls; default to 1 per 100 minutes.
/// </summary>
private const int LIST_ORDER_ITEMS_DEFAULT_THROTTLE_LIMIT = 10 * 60 * 1000;
private MarketplaceWebServiceOrders mwsService;
private string mwsSellerId;
private string[] mwsMarketplaceIdList;
private DateTime lastServiceCall = DateTime.MinValue;
private ProcessOrderHandler _processOrder;
/// <summary>
/// Event called when an order is received for processing.
/// </summary>
public event ProcessOrderHandler ProcessOrder
{
add { _processOrder += value; }
remove { _processOrder -= value; }
}
/// <summary>
/// Creates a new instance of the OrderFetcherSample class.
/// </summary>
/// <param name="service"></param>
public OrderFetcher(MarketplaceWebServiceOrders service, string sellerId, string[] marketplaceIdList)
{
mwsService = service;
mwsSellerId = sellerId;
mwsMarketplaceIdList = marketplaceIdList;
}
/// <summary>
/// Fetches all orders created between the starting time and the server's
/// local system time minus two minutes.
/// <param name="startTime">The starting time period of orders to fetch.</param>
public void FetchOrders(DateTime startTime)
{
FetchOrders(startTime, DateTime.MinValue);
}
/// <summary>
/// Fetches all orders created in the given time period and processes them locally.
/// <param name="startTime">The starting time period of orders to fetch.</param>
/// <param name="endTime">The ending time period of orders to fetch.</param>
public void FetchOrders(DateTime startTime, DateTime endTime)
{
ListOrdersRequest request = new ListOrdersRequest();
request.CreatedAfter = startTime;
if (endTime != DateTime.MinValue)
{
request.CreatedBefore = endTime;
}
request.SellerId = mwsSellerId;
request.MarketplaceId = new MarketplaceIdList();
request.MarketplaceId.Id = new List<string>();
foreach (string marketplaceId in mwsMarketplaceIdList)
{
request.MarketplaceId.Id.Add(marketplaceId);
}
List<Order> orderList = new List<Order>();
ListOrdersResponse response = null;
OrderFetcher.InvokeRetriable(LIST_ORDERS_DEFAULT_THROTTLE_LIMIT, delegate()
{
response = mwsService.ListOrders(request);
ProcessOrders(response.ListOrdersResult.Orders.Order);
});
String nextTokenString = response.ListOrdersResult.NextToken;
while (!string.IsNullOrEmpty(nextTokenString))
{
// If NextToken is set, continue looping through the orders.
ListOrdersByNextTokenRequest nextRequest = new ListOrdersByNextTokenRequest();
nextRequest.NextToken = nextTokenString;
nextRequest.SellerId = mwsSellerId;
ListOrdersByNextTokenResponse nextResponse = null;
OrderFetcher.InvokeRetriable(LIST_ORDERS_DEFAULT_THROTTLE_LIMIT, delegate()
{
nextResponse = mwsService.ListOrdersByNextToken(nextRequest);
ProcessOrders(nextResponse.ListOrdersByNextTokenResult.Orders.Order);
});
nextTokenString = nextResponse.ListOrdersByNextTokenResult.NextToken;
}
}
/// <summary>
/// Method called by the FetchOrders method to process the orders.
/// </summary>
/// <param name="orders">List of orders returned by FetchOrders</param>
protected virtual void ProcessOrders(List<Order> orders)
{
foreach (Order order in orders)
{
if (_processOrder != null)
{
_processOrder(order);
}
}
}
/// <summary>
/// Fetches the OrderItems for the specified orderId.
/// </summary>
public void FetchOrderItems(string orderId, ProcessOrderItemHandler handler)
{
if (handler == null) throw new ArgumentNullException("handler");
ListOrderItemsRequest request = new ListOrderItemsRequest();
request.SellerId = mwsSellerId;
request.AmazonOrderId = orderId;
ListOrderItemsResponse response = null;
OrderFetcher.InvokeRetriable(LIST_ORDER_ITEMS_DEFAULT_THROTTLE_LIMIT, delegate()
{
response = mwsService.ListOrderItems(request);
foreach (OrderItem orderItem in response.ListOrderItemsResult.OrderItems.OrderItem)
{
handler(orderItem);
}
});
String nextTokenString = response.ListOrderItemsResult.NextToken;
while (!string.IsNullOrEmpty(nextTokenString))
{
// If NextToken is set, continue looping through the orders.
ListOrderItemsByNextTokenRequest nextRequest = new ListOrderItemsByNextTokenRequest();
nextRequest.NextToken = nextTokenString;
nextRequest.SellerId = mwsSellerId;
ListOrderItemsByNextTokenResponse nextResponse = null;
OrderFetcher.InvokeRetriable(LIST_ORDER_ITEMS_DEFAULT_THROTTLE_LIMIT, delegate()
{
nextResponse = mwsService.ListOrderItemsByNextToken(nextRequest);
foreach (OrderItem orderItem in nextResponse.ListOrderItemsByNextTokenResult.OrderItems.OrderItem)
{
handler(orderItem);
}
});
nextTokenString = nextResponse.ListOrderItemsByNextTokenResult.NextToken;
}
}
/// <summary>
/// Invokes a method in a retriable fashion.
/// </summary>
/// <param name="throttledWaitTime">The amount of time to wait if the request is throttled.</param>
/// <param name="method">The method to invoke.</param>
public static void InvokeRetriable(RetriableMethodCall method)
{
InvokeRetriable(DEFAULT_THROTTLED_WAIT_TIMEOUT, method);
}
/// <summary>
/// Invokes a method in a retriable fashion.
/// </summary>
/// <param name="throttledWaitTime">The amount of time to wait if the request is throttled.</param>
/// <param name="method">The method to invoke.</param>
public static void InvokeRetriable(int throttledWaitTime, RetriableMethodCall method)
{
bool retryRequest = false;
do
{
retryRequest = false;
try
{
// Perform some action
method.Invoke();
}
catch (MarketplaceWebServiceOrdersException ordersErr)
{
// If the request is throttled, wait and try again.
if (ordersErr.ErrorCode == "RequestThrottled")
{
Console.WriteLine("Request is throttled; waiting...");
retryRequest = true;
System.Threading.Thread.Sleep(throttledWaitTime);
}
else
{
// On any other error, re-throw the exception to be handled by the caller
throw;
}
}
} while (retryRequest);
}
}
}
I think this may have to do with the way in which you imported (I assume) your web service objects. If it's a WCF service, make sure your "Model" is referenced along the different projects, and when adding the service reference, make sure to check the option "Reuse existing Types".
You have classes MarketplaceWebServiceOrders both in current FetchNewOrdersJob and in MarketplaceWebServiceOrders namespaces (also last namespace have same name as your class). Looks like your OrderFetcher class is declared in MarketplaceWebServiceOrders namespace. And it expects as argument MarketplaceWebServiceOrders which is also declared in that namespace. But you are trying to pass MarketplaceWebServiceOrders class, which is declared in current FetchNewOrdersJob namespace.
Rename your FetchNewOrdersJob.MarketplaceWebServiceOrders class to avoid these conflicts.

Regular expression to separate arguments in functions

I can't deal with a regular expression to separate the argument from function.
The function takes arguments in following way:
FunctionName(arg1;arg2;...;argn)
Now to make the rest of my code work I need to do the following-put every argument in ():
FunctionName((arg1);(arg2);(arg3))
The problem is that the arg can be anything- a number, an operator, other function
The test code for the solution is:
The function before regexp:
Function1((a1^5-4)/2;1/sin(a2);a3;a4)+Function2(a1;a2;1/a3)
After i needd to get sth like this:
Function1(((a1^5-4)/2);(1/sin(a2));(a3);(a4))+Function2((a1);(a2);(1/a3))
Unless I'm missing something, isn't it as simple as replacing ; with );( and surrounding the whole thing in ( ) ?
Using Regex:
(?:([^;()]+);?)+
and LINQ:
string result = "FunctionName(" +
String.Join(";",
from Capture capture in
Regex.Matches(inputString, #"FunctionName\((?:([^;()]+);?)+\)")[0].Groups[1].
Captures
select "(" + capture.Value + ")") + ")";
This is a far cry from a Regex but the potential for nested functions combined with the fact that this is a structured language being modified that a lexer/parser scheme is more appropriate.
Here is an example of a system that processes things of this nature
First, we define something that can be located in the input (the expression to modify)
public interface ISourcePart
{
/// <summary>
/// Gets the string representation of the kind of thing we're working with
/// </summary>
string Kind { get; }
/// <summary>
/// Gets the position this information is found at in the original source
/// </summary>
int Position { get; }
/// <summary>
/// Gets a representation of this data as Token objects
/// </summary>
/// <returns>An array of Token objects representing the data</returns>
Token[] AsTokens();
}
Next, we'll define a construct for housing tokens (identifiable portions of the source text)
public class Token : ISourcePart
{
public int Position { get; set; }
public Token[] AsTokens()
{
return new[] {this};
}
public string Kind { get; set; }
/// <summary>
/// Gets or sets the value of the token
/// </summary>
public string Value { get; set; }
/// <summary>
/// Creates a new Token
/// </summary>
/// <param name="kind">The kind (name) of the token</param>
/// <param name="match">The Match the token is to be generated from</param>
/// <param name="index">The offset from the beginning of the file the index of the match is relative to</param>
/// <returns>The newly created token</returns>
public static Token Create(string kind, Match match, int index)
{
return new Token
{
Position = match.Index + index,
Kind = kind,
Value = match.Value
};
}
/// <summary>
/// Creates a new Token
/// </summary>
/// <param name="kind">The kind (name) of the token</param>
/// <param name="value">The value to assign to the token</param>
/// <param name="position">The absolute position in the source file the value is located at</param>
/// <returns>The newly created token</returns>
public static Token Create(string kind, string value, int position)
{
return new Token
{
Kind = kind,
Value = value,
Position = position
};
}
}
We'll use Regexes to find our tokens in this example (below - Excerpt from Program.cs in my demo project).
/// <summary>
/// Breaks an input string into recognizable tokens
/// </summary>
/// <param name="source">The input string to break up</param>
/// <returns>The set of tokens located within the string</returns>
static IEnumerable<Token> Tokenize(string source)
{
var tokens = new List<Token>();
var sourceParts = new[] { new KeyValuePair<string, int>(source, 0) };
tokens.AddRange(Tokenize(OpenParen, "\\(", ref sourceParts));
tokens.AddRange(Tokenize(CloseParen, "\\)", ref sourceParts));
tokens.AddRange(Tokenize(Semi, ";", ref sourceParts));
tokens.AddRange(Tokenize(Operator, "[\\^\\\\*\\+\\-/]", ref sourceParts));
tokens.AddRange(Tokenize(Literal, "\\w+", ref sourceParts));
return tokens.OrderBy(x => x.Position);
}
As you can see, I've defined patterns for open and close parenthesis, semicolons, basic math operators and letters and numbers.
The Tokenize method is defined as follows (again from Program.cs in my demo project)
/// <summary>
/// Performs tokenization of a collection of non-tokenized data parts with a specific pattern
/// </summary>
/// <param name="tokenKind">The name to give the located tokens</param>
/// <param name="pattern">The pattern to use to match the tokens</param>
/// <param name="untokenizedParts">The portions of the input that have yet to be tokenized (organized as text vs. position in source)</param>
/// <returns>The set of tokens matching the given pattern located in the untokenized portions of the input, <paramref name="untokenizedParts"/> is updated as a result of this call</returns>
static IEnumerable<Token> Tokenize(string tokenKind, string pattern, ref KeyValuePair<string, int>[] untokenizedParts)
{
//Do a bit of setup
var resultParts = new List<KeyValuePair<string, int>>();
var resultTokens = new List<Token>();
var regex = new Regex(pattern);
//Look through all of our currently untokenized data
foreach (var part in untokenizedParts)
{
//Find all of our available matches
var matches = regex.Matches(part.Key).OfType<Match>().ToList();
//If we don't have any, keep the data as untokenized and move to the next chunk
if (matches.Count == 0)
{
resultParts.Add(part);
continue;
}
//Store the untokenized data in a working copy and save the absolute index it reported itself at in the source file
var workingPart = part.Key;
var index = part.Value;
//Look through each of the matches that were found within this untokenized segment
foreach (var match in matches)
{
//Calculate the effective start of the match within the working copy of the data
var effectiveStart = match.Index - (part.Key.Length - workingPart.Length);
resultTokens.Add(Token.Create(tokenKind, match, part.Value));
//If we didn't match at the beginning, save off the first portion to the set of untokenized data we'll give back
if (effectiveStart > 0)
{
var value = workingPart.Substring(0, effectiveStart);
resultParts.Add(new KeyValuePair<string, int>(value, index));
}
//Get rid of the portion of the working copy we've already used
if (match.Index + match.Length < part.Key.Length)
{
workingPart = workingPart.Substring(effectiveStart + match.Length);
}
else
{
workingPart = string.Empty;
}
//Update the current absolute index in the source file we're reporting to be at
index += effectiveStart + match.Length;
}
//If we've got remaining data in the working copy, add it back to the untokenized data
if (!string.IsNullOrEmpty(workingPart))
{
resultParts.Add(new KeyValuePair<string, int>(workingPart, index));
}
}
//Update the untokenized data to contain what we couldn't process with this pattern
untokenizedParts = resultParts.ToArray();
//Return the tokens we were able to extract
return resultTokens;
}
Now that we've got the methods and types in place to handle our tokenized data, we need to be able to recognize pieces of larger meaning, like calls to simple functions (like sin(x)), complex functions (like Function1(a1;a2;a3)), basic mathematical operations (like +, -, *, etc.), and so on. We'll make a simple parser for dealing with that; firstly we'll define a match condition for a parse node.
public class ParseNodeDefinition
{
/// <summary>
/// The set of parse node definitions that could be transitioned to from this one
/// </summary>
private readonly IList<ParseNodeDefinition> _nextNodeOptions;
/// <summary>
/// Creates a new ParseNodeDefinition
/// </summary>
private ParseNodeDefinition()
{
_nextNodeOptions = new List<ParseNodeDefinition>();
}
/// <summary>
/// Gets whether or not this definition is an acceptable ending point for the parse tree
/// </summary>
public bool IsValidEnd { get; private set; }
/// <summary>
/// Gets the name an item must have for it to be matched by this definition
/// </summary>
public string MatchItemsNamed { get; private set; }
/// <summary>
/// Gets the set of parse node definitions that could be transitioned to from this one
/// </summary>
public IEnumerable<ParseNodeDefinition> NextNodeOptions
{
get { return _nextNodeOptions; }
}
/// <summary>
/// Gets or sets the tag that will be associated with the data if matched
/// </summary>
public string Tag { get; set; }
/// <summary>
/// Creates a new ParseNodeDefinition matching items with the specified name/kind.
/// </summary>
/// <param name="matchItemsNamed">The name of the item to be matched</param>
/// <param name="tag">The tag to associate with matched items</param>
/// <param name="isValidEnd">Whether or not the element is a valid end to the parse tree</param>
/// <returns>A ParseNodeDefinition capable of matching items of the given name</returns>
public static ParseNodeDefinition Create(string matchItemsNamed, string tag, bool isValidEnd)
{
return new ParseNodeDefinition { MatchItemsNamed = matchItemsNamed, Tag = tag, IsValidEnd = isValidEnd };
}
public ParseNodeDefinition AddOption(string matchItemsNamed)
{
return AddOption(matchItemsNamed, string.Empty, false);
}
public ParseNodeDefinition AddOption(string matchItemsNamed, string tag)
{
return AddOption(matchItemsNamed, tag, false);
}
/// <summary>
/// Adds an option for a named node to follow this one in the parse tree the node is a part of
/// </summary>
/// <param name="matchItemsNamed">The name of the item to be matched</param>
/// <param name="tag">The tag to associate with matched items</param>
/// <param name="isValidEnd">Whether or not the element is a valid end to the parse tree</param>
/// <returns>The ParseNodeDefinition that has been added</returns>
public ParseNodeDefinition AddOption(string matchItemsNamed, string tag, bool isValidEnd)
{
var node = Create(matchItemsNamed, tag, isValidEnd);
_nextNodeOptions.Add(node);
return node;
}
public ParseNodeDefinition AddOption(string matchItemsNamed, bool isValidEnd)
{
return AddOption(matchItemsNamed, string.Empty, isValidEnd);
}
/// <summary>
/// Links the given node as an option for a state to follow this one in the parse tree this node is a part of
/// </summary>
/// <param name="next">The node to add as an option</param>
public void LinkTo(ParseNodeDefinition next)
{
_nextNodeOptions.Add(next);
}
}
This will let us match a single element by name (whether it's a ParseTree defined later) or a Token as they both implement the ISourcePart interface. Next we'll make a ParseTreeDefinition that allows us to specify sequences of ParseNodeDefinitions for matching.
public class ParseTreeDefinition
{
/// <summary>
/// The set of parse node definitions that constitute an initial match to the parse tree
/// </summary>
private readonly IList<ParseNodeDefinition> _initialNodeOptions;
/// <summary>
/// Creates a new ParseTreeDefinition
/// </summary>
/// <param name="name">The name to give to parse trees generated from full matches</param>
public ParseTreeDefinition(string name)
{
_initialNodeOptions = new List<ParseNodeDefinition>();
Name = name;
}
/// <summary>
/// Gets the set of parse node definitions that constitute an initial match to the parse tree
/// </summary>
public IEnumerable<ParseNodeDefinition> InitialNodeOptions { get { return _initialNodeOptions; } }
/// <summary>
/// Gets the name of the ParseTreeDefinition
/// </summary>
public string Name { get; private set; }
/// <summary>
/// Adds an option for a named node to follow this one in the parse tree the node is a part of
/// </summary>
/// <param name="matchItemsNamed">The name of the item to be matched</param>
/// <returns>The ParseNodeDefinition that has been added</returns>
public ParseNodeDefinition AddOption(string matchItemsNamed)
{
return AddOption(matchItemsNamed, string.Empty, false);
}
/// <summary>
/// Adds an option for a named node to follow this one in the parse tree the node is a part of
/// </summary>
/// <param name="matchItemsNamed">The name of the item to be matched</param>
/// <param name="tag">The tag to associate with matched items</param>
/// <returns>The ParseNodeDefinition that has been added</returns>
public ParseNodeDefinition AddOption(string matchItemsNamed, string tag)
{
return AddOption(matchItemsNamed, tag, false);
}
/// <summary>
/// Adds an option for a named node to follow this one in the parse tree the node is a part of
/// </summary>
/// <param name="matchItemsNamed">The name of the item to be matched</param>
/// <param name="tag">The tag to associate with matched items</param>
/// <param name="isValidEnd">Whether or not the element is a valid end to the parse tree</param>
/// <returns>The ParseNodeDefinition that has been added</returns>
public ParseNodeDefinition AddOption(string matchItemsNamed, string tag, bool isValidEnd)
{
var node = ParseNodeDefinition.Create(matchItemsNamed, tag, isValidEnd);
_initialNodeOptions.Add(node);
return node;
}
/// <summary>
/// Adds an option for a named node to follow this one in the parse tree the node is a part of
/// </summary>
/// <param name="matchItemsNamed">The name of the item to be matched</param>
/// <param name="isValidEnd">Whether or not the element is a valid end to the parse tree</param>
/// <returns>The ParseNodeDefinition that has been added</returns>
public ParseNodeDefinition AddOption(string matchItemsNamed, bool isValidEnd)
{
return AddOption(matchItemsNamed, string.Empty, isValidEnd);
}
/// <summary>
/// Attempts to follow a particular branch in the parse tree from a given starting point in a set of source parts
/// </summary>
/// <param name="parts">The set of source parts to attempt to match in</param>
/// <param name="startIndex">The position to start the matching attempt at</param>
/// <param name="required">The definition that must be matched for the branch to be followed</param>
/// <param name="nodes">The set of nodes that have been matched so far</param>
/// <returns>true if the branch was followed to completion, false otherwise</returns>
private static bool FollowBranch(IList<ISourcePart> parts, int startIndex, ParseNodeDefinition required, ICollection<ParseNode> nodes)
{
if (parts[startIndex].Kind != required.MatchItemsNamed)
{
return false;
}
nodes.Add(new ParseNode(parts[startIndex], required.Tag));
return parts.Count > (startIndex + 1) && required.NextNodeOptions.Any(x => FollowBranch(parts, startIndex + 1, x, nodes)) || required.IsValidEnd;
}
/// <summary>
/// Attempt to match the parse tree definition against a set of source parts
/// </summary>
/// <param name="parts">The source parts to match against</param>
/// <returns>true if the parse tree was matched, false otherwise. parts is updated by this method to consolidate matched nodes into a ParseTree</returns>
public bool Parse(ref IList<ISourcePart> parts)
{
var partsCopy = parts.ToList();
for (var i = 0; i < parts.Count; ++i)
{
var tree = new List<ParseNode>();
if (InitialNodeOptions.Any(x => FollowBranch(partsCopy, i, x, tree)))
{
partsCopy.RemoveRange(i, tree.Count);
partsCopy.Insert(i, new ParseTree(Name, tree.ToArray(), tree[0].Position));
parts = partsCopy;
return true;
}
}
return false;
}
}
Of course these don't do us much good without having some place to store the results of the matchers we've defined so far, so let's define ParseTree and ParseNode where a ParseTree is simply a collection of ParseNode objects where ParseNode is a wrapper around a ParseTree or Token (or more generically any ISourcePart).
public class ParseTree : ISourcePart
{
/// <summary>
/// Creates a new ParseTree
/// </summary>
/// <param name="kind">The kind (name) of tree this is</param>
/// <param name="nodes">The nodes the tree matches</param>
/// <param name="position">The position in the source file this tree is located at</param>
public ParseTree(string kind, IEnumerable<ISourcePart> nodes, int position)
{
Kind = kind;
ParseNodes = nodes.ToList();
Position = position;
}
public string Kind { get; private set; }
public int Position { get; private set; }
/// <summary>
/// Gets the nodes that make up this parse tree
/// </summary>
public IList<ISourcePart> ParseNodes { get; internal set; }
public Token[] AsTokens()
{
return ParseNodes.SelectMany(x => x.AsTokens()).ToArray();
}
}
public class ParseNode : ISourcePart
{
/// <summary>
/// Creates a new ParseNode
/// </summary>
/// <param name="sourcePart">The data that was matched to create this node</param>
/// <param name="tag">The tag data (if any) associated with the node</param>
public ParseNode(ISourcePart sourcePart, string tag)
{
SourcePart = sourcePart;
Tag = tag;
}
public string Kind { get { return SourcePart.Kind; } }
/// <summary>
/// Gets the tag associated with the matched data
/// </summary>
public string Tag { get; private set; }
/// <summary>
/// Gets the data that was matched to create this node
/// </summary>
public ISourcePart SourcePart { get; private set; }
public int Position { get { return SourcePart.Position; } }
public Token[] AsTokens()
{
return SourcePart.AsTokens();
}
}
That's it for the constructs we need, so we'll move into configuring our parse tree definitions. The code from here on is from Program.cs in my demo.
As you might have noticed in the block above about declaring the patterns for each token, there were some values referenced but not defined, here they are.
private const string CloseParen = "CloseParen";
private const string ComplexFunctionCall = "ComplexFunctionCall";
private const string FunctionCallStart = "FunctionCallStart";
private const string Literal = "Literal";
private const string OpenParen = "OpenParen";
private const string Operator = "Operator";
private const string ParenthesisRequiredElement = "ParenthesisRequiredElement";
private const string ParenthesizedItem = "ParenthesizedItem";
private const string Semi = "Semi";
private const string SimpleFunctionCall = "SimpleFunctionCall";
Let's begin by defining a pattern that matches literals (\w+ pattern) that are followed by open parenthesis; we'll use this to match things like sin( or Function1(.
static ParseTreeDefinition CreateFunctionCallStartTree()
{
var tree = new ParseTreeDefinition(FunctionCallStart);
var name = tree.AddOption(Literal);
name.AddOption(OpenParen, true);
return tree;
}
Really not a whole lot to it, setup a tree, add an option for the first thing to match as a Literal, add an option of the next thing to match as an open parenthesis and say that it can end the parse tree.
Now for one that's a little more complex, binary mathematical operations (couldn't think of any unary operations that would need to be included)
static ParseTreeDefinition CreateBinaryOperationResultTree()
{
var tree = new ParseTreeDefinition(Literal);
var parenthesizedItem = tree.AddOption(ParenthesizedItem);
var literal = tree.AddOption(Literal);
var simpleCall = tree.AddOption(SimpleFunctionCall);
var complexCall = tree.AddOption(ComplexFunctionCall);
var #operator = parenthesizedItem.AddOption(Operator);
literal.LinkTo(#operator);
simpleCall.LinkTo(#operator);
complexCall.LinkTo(#operator);
#operator.AddOption(ParenthesizedItem, true);
#operator.AddOption(Literal, true);
#operator.AddOption(SimpleFunctionCall, true);
#operator.AddOption(ComplexFunctionCall, true);
return tree;
}
Here we say that the parse tree can start with a parenthesized item (like (1/2)), a literal (like a5 or 3), a simple call (like sin(4)) or a complex one (like Function1(a1;a2;a3)). In essence we've just defined the options for the left hand operand. Next, we say that the parenthesized item must be followed by an Operator (one of the mathematical operators from the pattern declared way up at the beginning) and, for convenience, we'll say that all of the other options for the left hand operand can progress to that same state (having the operator). Next, the operator must have a right hand side as well, so we give it a duplicate set of options to progress to. Note that they are not the same definitions as the left hand operands, these have the flag set to be able to terminate the parse tree. Notice that the parse tree is named Literal to avoid having to specify yet another kind of element to match all over the place.
Next up, parenthesized items:
static ParseTreeDefinition CreateParenthesizedItemTree()
{
var tree = new ParseTreeDefinition(ParenthesizedItem);
var openParen = tree.AddOption(OpenParen);
var nestedSimpleCall = openParen.AddOption(SimpleFunctionCall);
var nestedComplexCall = openParen.AddOption(ComplexFunctionCall);
var arg = openParen.AddOption(Literal);
var parenthesizedItem = openParen.AddOption(ParenthesizedItem);
var closeParen = nestedSimpleCall.AddOption(CloseParen, true);
arg.LinkTo(closeParen);
parenthesizedItem.LinkTo(closeParen);
nestedComplexCall.LinkTo(closeParen);
return tree;
}
Nice and easy with this one, start with a parenthesis, follow it up with pretty much anything, follow that with another parenthesis to close it.
Simple calls (like sin(x))
static ParseTreeDefinition CreateSimpleFunctionCallTree()
{
var tree = new ParseTreeDefinition(SimpleFunctionCall);
var openParen = tree.AddOption(FunctionCallStart);
var nestedItem = openParen.AddOption(ParenthesizedItem);
var nestedSimpleCall = openParen.AddOption(SimpleFunctionCall);
var nestedComplexCall = openParen.AddOption(ComplexFunctionCall);
var arg = openParen.AddOption(Literal);
var parenthesizedItem = openParen.AddOption(ParenthesizedItem);
var closeParen = nestedSimpleCall.AddOption(CloseParen, true);
arg.LinkTo(closeParen);
nestedItem.LinkTo(closeParen);
parenthesizedItem.LinkTo(closeParen);
nestedComplexCall.LinkTo(closeParen);
return tree;
}
Complex calls (like Function1(a1;a2;a3))
static ParseTreeDefinition CreateComplexFunctionCallTree()
{
var tree = new ParseTreeDefinition(ComplexFunctionCall);
var openParen = tree.AddOption(FunctionCallStart);
var arg = openParen.AddOption(Literal, ParenthesisRequiredElement);
var simpleCall = openParen.AddOption(SimpleFunctionCall, ParenthesisRequiredElement);
var complexCall = openParen.AddOption(ComplexFunctionCall, ParenthesisRequiredElement);
var nested = openParen.AddOption(ParenthesizedItem);
var semi = arg.AddOption(Semi);
simpleCall.LinkTo(semi);
complexCall.LinkTo(semi);
nested.LinkTo(semi);
var arg2 = semi.AddOption(Literal, ParenthesisRequiredElement);
var simpleCall2 = semi.AddOption(SimpleFunctionCall, ParenthesisRequiredElement);
var complexCall2 = semi.AddOption(ComplexFunctionCall, ParenthesisRequiredElement);
var nested2 = semi.AddOption(ParenthesizedItem);
arg2.LinkTo(semi);
simpleCall2.LinkTo(semi);
complexCall2.LinkTo(semi);
nested2.LinkTo(semi);
var closeParen = arg2.AddOption(CloseParen, true);
arg2.LinkTo(closeParen);
simpleCall2.LinkTo(closeParen);
complexCall2.LinkTo(closeParen);
return tree;
}
That's all the trees we'll need, so let's take a look at the code that runs this all
static void Main()
{
//The input string
const string input = #"Function1((a1^5-4)/2;1/sin(a2);a3;a4)+Function2(a1;a2;1/a3)";
//Locate the recognizable tokens within the source
IList<ISourcePart> tokens = Tokenize(input).Cast<ISourcePart>().ToList();
//Create the parse trees we'll need to be able to recognize the different parts of the input
var functionCallStartTree = CreateFunctionCallStartTree();
var parenthethesizedItemTree = CreateParenthesizedItemTree();
var simpleFunctionCallTree = CreateSimpleFunctionCallTree();
var complexFunctionCallTree = CreateComplexFunctionCallTree();
var binaryOpTree = CreateBinaryOperationResultTree();
//Parse until we can't parse anymore
while (functionCallStartTree.Parse(ref tokens) || binaryOpTree.Parse(ref tokens) || parenthethesizedItemTree.Parse(ref tokens) || simpleFunctionCallTree.Parse(ref tokens) || complexFunctionCallTree.Parse(ref tokens))
{ }
//Run our post processing to fix the parenthesis in the input
FixParenthesis(ref tokens);
//Collapse our parse tree(s) back to a string
var values = tokens.OrderBy(x => x.Position).SelectMany(x => x.AsTokens()).Select(x => x.Value);
//Print out our results and wait
Console.WriteLine(string.Join(string.Empty, values));
Console.ReadLine();
}
The only thing we've got left to define is how to actually do the wrapping of the elements in the argument list of a "complex" call. That's handled by the FixParenthesis method.
private static void FixParenthesis(ref IList<ISourcePart> items)
{
//Iterate through the set we're examining
for (var i = 0; i < items.Count; ++i)
{
var parseNode = items[i] as ParseNode;
//If we've got a parse node...
if (parseNode != null)
{
var nodeTree = parseNode.SourcePart as ParseTree;
//If the parse node represents a parse tree...
if (nodeTree != null)
{
//Fix parenthesis within the tree
var nodes = nodeTree.ParseNodes;
FixParenthesis(ref nodes);
nodeTree.ParseNodes = nodes;
}
//If this parse node required parenthesis, replace the subtree and add them
if (parseNode.Tag == ParenthesisRequiredElement)
{
var nodeContents = parseNode.AsTokens();
var combined = string.Join(string.Empty, nodeContents.OrderBy(x => x.Position).Select(x => x.Value));
items[i] = Token.Create(parseNode.Kind, string.Format("({0})", combined), parseNode.Position);
}
continue;
}
var parseTree = items[i] as ParseTree;
//If we've got a parse tree...
if (parseTree != null)
{
//Fix parenthesis within the tree
var nodes = parseTree.ParseNodes;
FixParenthesis(ref nodes);
parseTree.ParseNodes = nodes;
}
}
}
At any rate, I hope this has helped or at least provided a fun diversion.
I probably managed to deal with it(now testing). It turned out to be 5-stage operation. Assuming that '{' and ';' cannot occur in function I've done sth like this:
sBuffer = Regex.Replace(sBuffer, #"(?<sep>[;])", "};{");
sBuffer = Regex.Replace(sBuffer, #"([(])(?<arg>.+?)[}]", "({${arg}}");
sBuffer = Regex.Replace(sBuffer, #"([;])(?<arg>.+?)([)]){1}", ";${arg}})");
sBuffer = Regex.Replace(sBuffer, #"{", "(");
sBuffer = Regex.Replace(sBuffer, #"}", ")");
0.
function1((a1^5-4)/2;1/sin(a2);a3;a4)+function2(a1;a2;1/a3)'
1.First line replaces ; with };{
function1((a1^5-4)/2};{1/sin(a2)};{a3};{a4)+function2(a1};{a2};{1/a3)
2.For first argument - after ( or (not intended) arguments which contain ')' replace (arg};with ({arg}:
function1({(a1^5-4)/2};{1/sin({a2)};{a3};{a4)+function2({a1};{a2};{1/a3)
3. The same at the and of function: {arg) with {arg}:
function1({(a1^5-4)/2};{1/sin({a2})};{a3};{a4})+function2({a1};{a2};{1/a3})
4.5. Replace '{' and '}' with '(' ')':
function1(((a1^5-4)/2);(1/sin((a2)));(a3);(a4))+function2((a1);(a2);(1/a3))
We have some extra () specially when argument itself is surrounded by '(' ')' (nested function) but it doesn't metter as the code is then proceed by Reversed Polish Notation
This is my first code for regexp(I found out about rgexp just few days ago- I'm a beginer) . I hope it's satisfies all the cases (at least those that can occur in excel formulas)

Categories