How to free up memory from IEnumerable - c#

I'm making a program that prints out all Nodes in a Collatz tree in C#.
This takes up a whole lot of memory, and outOfMemoryExceptions have been the word of the day. The base of the program is as follows:
var fileStream = File.Create("collatz.txt"); //just print into a textfile for testing
var writer = new StreamWriter(fileStream);
IEnumerable<Node> nodeList = new List<Node>()
{
new Node(1, null) //rootNode
};
Task flushTask = Task.CompletedTask;
while (nodeList.Any())
{
var tempList = CalcChildren(nodeList); //Return a IEnumerable of children of all the parents in the nodeList
await flushTask;
foreach (var node in tempList)
writer.WriteLine($"{node.Value} -> {node.Parent.Value}: {node.StepsFromRoot}");
flushTask = writer.FlushAsync();
nodeList = tempList;
}
writer.Close();
static IEnumerable<Node> CalcChildren(IEnumerable<Node> parents)
{
foreach(var parent in parents)
foreach (var child in CalcChildren(parent))
yield return child;
}
static IEnumerable<Node> CalcChildren(Node parent)
{
var multiValue = parent.CalcMultiplicationValue();
if (multiValue.HasValue)
{
var child = new Node(multiValue.Value, parent);
parent.MultiplicationChild = child;
yield return child;
}
var divValue = parent.CalcDivisionValue();
if (divValue.HasValue && divValue.Value!=1)
{
var child = new Node(divValue.Value, parent);
parent.DivisionChild = child;
yield return child;
}
}
I think herein lies the problem but for the sake of completeness, the Node class:
public class Node
{
public Node(int value, Node parent)
{
Value = value;
Parent = parent;
if (parent != null)
StepsFromRoot = parent.StepsFromRoot+1;
else
StepsFromRoot = 0;
}
public int Value { get; }
public Node Parent { get; set; }
public Node DivisionChild { get; set; }
public Node MultiplicationChild { get; set; }
public int StepsFromRoot { get; set; }
public int? CalcMultiplicationValue()
{
if(Value<=int.MaxValue/2)
return 2 * Value;
return null;
}
public int? CalcDivisionValue()
{
double newValue = (Value - 1) / 3.0;
if (newValue % 2 == 1 && newValue >= 1)
return (int)newValue;
return null;
}
}
I'm trying my best to drop as much memory as possible, but it doesn't seem to work. Around the 105th iteration this program takes up 4 GB of memory! I first thought it was because the buffer of my writer was taking up all the memory. Now I think it's because the collection of nodes isn't cleaned up by the garbage collector.
I tried to make it more minimal. But I'm not sure what to delete while keeping it complete. Thanks in advance!

Related

How to find the next node in a tree?

The problem goes like this:
We have a tree that is built using the class Node where an instance of the class represents a node in the tree. For simplicity, the node has a single data field of type int.
Your task is to write the extension method NodeExtensions.Next() to find the next element in the tree. You can write as many helper methods as you want, but don't change the signature of the extension method NodeExtensions.Next().
I have this Node class:
public class Node
{
private List<Node> _children;
public Node(int data, params Node[] nodes)
{
Data = data;
AddRange(nodes);
}
public Node Parent { get; set; }
public IEnumerable<Node> Children
{
get
{
return _children != null
? _children
: Enumerable.Empty<Node>();
}
}
public int Data { get; private set; }
public void Add(Node node)
{
Debug.Assert(node.Parent == null);
if (_children == null)
{
_children = new List<Node>();
}
_children.Add(node);
node.Parent = this;
}
public void AddRange(IEnumerable<Node> nodes)
{
foreach (var node in nodes)
{
Add(node);
}
}
public override string ToString()
{
return Data.ToString();
}
}
The solution should be a extension method such as this
public static Node Next(this Node node)
{
}
The code I tried:
public static Node Next(this Node node)
{
var newNode = NextLargerElement(node, node.Data);
return newNode;
}
public static Node res;
public static Node NextLargerElementUtil(Node root, int x)
{
if (root == null)
return null;
if (root.Data > x)
if ((res == null || (res).Data > root.Data))
res = root;
foreach (var children in root.Children)
{
NextLargerElementUtil(children, x);
}
return res;
}
static Node NextLargerElement(Node root, int x)
{
res = null;
NextLargerElementUtil(root, x);
return res;
}
Here is a testing case:
[Test]
public void Test()
{
// Test tree:
//
// 1
// +-2
// +-3
// +-4
// +-5
// +-6
// +-7
//
var root = new Node(
1,
new Node(
2,
new Node(3),
new Node(4)),
new Node(
5,
new Node(6),
new Node(7)));
// Expected output:
//
// 1
// 2
// 3
// 4
// 5
// 6
// 7
//
var n = root;
while (n != null)
{
Console.WriteLine(n.Data);
n = n.Next();
}
// Test
//
n = root;
Assert.AreEqual(1, n.Data);
n = n.Next();
Assert.AreEqual(2, n.Data);
n = n.Next();
Assert.AreEqual(3, n.Data);
n = n.Next();
Assert.AreEqual(4, n.Data);
n = n.Next();
Assert.AreEqual(5, n.Data);
n = n.Next();
Assert.AreEqual(6, n.Data);
n = n.Next();
Assert.AreEqual(7, n.Data);
n = n.Next();
Assert.IsNull(n);
}
You can use Equals to backtrack your current node position in the tree and return it's parent's next child, if any, if there is not one then you need to backtrack the current node parent position and so on:
public static Node Next(this Node node)
{
if(node.Children != null && node.Children.Any())
{
return node.Children.First();
}
// "backtracking", also can be done recursively
var parent = node.Parent;
while(parent != null)
{
var returnNext = false; // return next element in Children if current is node
foreach (var element in parent.Children)
{
if(returnNext)
{
return element;
}
if(element == node)
{
returnNext = true;
node = parent; // to find parent's position if there is no next child
}
}
parent = parent.Parent;
}
return null;
}

How to avoid out of memory using trie by C#

how to write billions of data into a trie with less memory
I want to extract some infomation from news like company names,so I write billions of company names into a trie,but it needs much memory and throw out of memory exception,I don't know how to solve it,so anyone can help,thanks in advance.
public class Node
{
public char Value { get; set; }
public List<Node> Children { get; set; }
public int Depth { get; set; }
public string Code { get; set; }
public bool Terminal { get; set; }
public Node(char value, int depth)
{
Value = value;
Depth = depth;
Children = new List<Node>();
}
public Node FindChildNode(char c)
{
foreach (var child in Children)
if (child.Value == c)
return child;
return null;
}
}
public class Trie
{
private Node _root;
public Trie()
{
_root = new Node('^',0);
}
public Node Prefix(string s)
{
var currentNode = _root;
var result = currentNode;
foreach (var c in s)
{
currentNode = currentNode.FindChildNode(c);
if (currentNode == null)
break;
result = currentNode;
}
return result;
}
public void Insert(string randomLength,string code)
{
var commonPrefix = Prefix(randomLength);
var current = commonPrefix;
for (var i = current.Depth; i < s.Length; i++)
{
var newNode = new Node(s[i], current.Depth + 1);
if (i+1==s.Length)
{
newNode.Terminal = true;
newNode.Code = code;
}
current.Children.Add(newNode);
current = newNode;
}
}
}
Trie t=new Trie();
t.Insert("C","ABCG00DFD");
The aboved statement run 1000000000 Loops and the "C" can be replaced with different string with different length,as the loops increasing,it throw out of memory exception,so how to avoid or change it?
Have a go at this Trie and see if you can get it to work for what you need:
public class Trie : Dictionary<char, Trie>
{
public void Add(string value)
{
var c = String.IsNullOrEmpty(value) ? '\0' : value[0];
if (!this.ContainsKey(c))
{
this[c] = new Trie();
}
if (c != '\0')
{
this[c].Add(value.Substring(1));
}
}
}

Removing cloned nodes from list

I am making a filtereditems class, that will be displayed as a treeview in WPF. The filtereditems class contains only certain node items from a treeitems class that contain certain criteria. I am able clone all of the tree items and add them to the filtereditems list. From there I find nodes that do not meet the criteria and remove them appropriately. However I have found that using the clones renders me unable to remove these items. Is there something I should know about cloned items and why they can't be removed from my collection?
public class Node: INotifyPropertyChanged, ICloneable
{
public Name { get;set;}
public ID {get;set;}
public ParentNode {get;set;}
public ObservableCollection<Nodes> ChildNodes{get;set;}
public object Clone()
{
Node toReturn = new Node();
toReturn.Name = this.Name;
toReturn.ID = this.ID;
toReturn.ParentNode = this.ParentNode;
foreach (Node child in this.ChildNodes)
{
toReturn.ChildNodes.Add((Node) child.Clone());
}
return toReturn;
}
}
public void filterStart(ChildNodesListViewDataSource _filterStart)
{
if (this.FilterString != null && this.Entity != null)
{
this.TotalItemsNumber = 0;
this.FilterItemsNumber = 0;
this.FilterTreeItems.Clear();
foreach (Node y in TreeItems)
{
this.FilterTreeItems.Add((Node)y.Clone());
foreach (Node x in FilterTreeItems)
{
FilterRoot(x);
}
}
TakeOutTrash();
public bool FilterRoot(Node FilterItems)
{
bool HasMatchingChildren = false;
if (FilterItems.ChildNodes != null || FilterItems.ChildNodes.Count !=0)
{
foreach (Node FilterItemsComponenents in FilterItems.ChildNodes)
{
if (FilterRoot(FilterItemsComponenents))
{
HasMatchingChildren = true;
}
}
}
string NameOfFilterItem = FilterItems.Name.ToUpper();
string FilterStringUpperCase = FilterString.ToUpper();
bool FilterStringCheck = NameOfFilterItem.Contains(FilterStringUpperCase);
if (!FilterStringCheck && !HasMatchingChildren)
{
trimIDs.TrashCan.Add(FilterItems);
return false;
}
else
{
return true;
}
}
public void TakeOutTrash()
{
foreach (Node node in trimIDs.TrashCan)
{
this.FilterTreeItems.Remove(node);
}
}
public class TrimIDs
{
public IList<ComponentNodeViewModel> TrashCan { get; set;}
{
TrashCan = new List<ComponentNodeViewModel>();
}
}
I actually solved my issue by creating another clone method that clones anything Node that is not in the List of Nodes not to clone: (Hope this helps anyone that was in my same predicament).
public object Clone(IList<Node> ListNotToClone)
{
NodetoReturnFiltered = new Node();
toReturnFiltered.Name = this.Name;
toReturnFiltered.ID = this.ID;
toReturnFiltered.ParentNode= this.ParentNode;
foreach (Node child in this.ComponentNodes)
{
if (!ListNotToClone.Contains(child))
{
toReturnFiltered.ComponentNodes.Add((Node)child.Clone(ListNotToClone));
}
}
return toReturnFiltered;
}
I then used this method :
public void TakeOutTrash()
{
foreach (ComponentNodeViewModel root in this.FilterTreeItems)
{
FilterHolder = (ComponentNodeViewModel)root.Clone(trimIDs.TrashCan);
}
}
And passed FilterHolder back into FilterTreeItems in the FilterStart method,
this.FilterTreeItems.Clear();
this.FilterTreeItems.Add(FilterHolder);

How to get parent node in a tree structure like this

How it would be possible to get a parent when tree structure is like this:
public class TreeModel
{
public int ID { get; set; }
public List<TreeModel> Children { get; set; }
}
Let's say we can't add a parent element item to this class (public TreeModel Parent { get; set; }).
Edit
How to get element m22 (ID=22) parent m2 (ID=2) from the m1? I thought we could iterate through m1 and somehow return parent when condition is right.
var m1 = new TreeModel() { ID = 1 };
var m2 = new TreeModel() { ID = 2 };
var m21 = new TreeModel() { ID = 21 };
var m22 = new TreeModel() { ID = 22 };
var m3 = new TreeModel() { ID = 3 };
m1.Children.Add(m2);
m2.Children.Add(m21);
m2.Children.Add(m22);
m1.Children.Add(m3);
var parent = m1.GetParent(p => p.ID == 22); //<-- How?
public IEnumerable<TreeModel> GetAllDescendants(IEnumerable<TreeModel> rootNodes)
{
var descendants = rootNodes.SelectMany(_ => GetAllDescendants(_.Children));
return rootNodes.Concat(descendants);
}
public static TreeModel GetParent(this TreeModel rootNode, Func<TreeModel, bool> childSelector)
{
var allNodes = GetAllDescendants(new [] { rootNode });
var parentsOfSelectedChildren = allNodes.Where(node => node.Children.Any(childSelector));
return parentsOfSelectedChildren.Single();
}
m1.GetParent(_ => _.ID == 22);
Obtain a flat list of all nodes
Search this list for the node whose direct children contains m22
Use this code pattern. It simplifies the code because you don't have to explicitly add nodes to the children and each node knows who its parent is and who its children are. Also it is all type safe.
class Program
{
static void Main(string[] args)
{
var m1=new TreeModel() { ID=1 };
var m2=new TreeModel(m1) { ID=2 };
var m21=new TreeModel(m2) { ID=21 };
var m22=new TreeModel(m2) { ID=22};
var m3=new TreeModel(m1) { ID=3 };
var item=m1.RecursiveFind((p) => p.ID==22);
var parent=item.Parent;
// parent.ID == 2
var root=item.Root;
// root.ID == 1;
}
}
public class TreeModel : Tree<TreeModel>
{
public int ID { get; set; }
public TreeModel() { }
public TreeModel(TreeModel parent) : base(parent) { }
}
public class Tree<T> where T : Tree<T>
{
protected Tree() : this(null) { }
protected Tree(T parent)
{
Parent=parent;
Children=new List<T>();
if(parent!=null)
{
parent.Children.Add(this as T);
}
}
public T Parent { get; set; }
public List<T> Children { get; set; }
public bool IsRoot { get { return Parent==null; } }
public T Root { get { return IsRoot?this as T:Parent.Root; } }
public T RecursiveFind(Predicate<T> check)
{
if(check(this as T)) return this as T;
foreach(var item in Children)
{
var result=item.RecursiveFind(check);
if(result!=null)
{
return result;
}
}
return null;
}
}
When you derive from Tree<T>, you create custom tree structures that you design what the node class is (TreeModel here) and how to handle parents, children and siblings if needed.
What about:
public class SaneTreeModel: TreeModel
{
public SaneTreeModel Parent { get; set; }
}
}
I would approach this by first finding the first element that satisfies the condition (ID == 22 in your example) and then finding the parent of this element. Not the best solution but maybe you will need them separately for something else.
public TreeModel GetParent(Func<TreeModel, bool> function)
{
return GetParent(Where(function));
}
private TreeModel GetParent(TreeModel treeModel)
{
if (Children == null) return null;
if (Children.Contains(treeModel)) return this;
foreach (TreeModel child in Children)
{
TreeModel result = child.GetParent(treeModel);
if (result != null)
return result;
}
return null;
}
private TreeModel Where(Func<TreeModel, bool> function)
{
if (Children == null) return null;
foreach (TreeModel child in Children)
{
if (function(child))
return child;
TreeModel result = child.Where(function);
if (result != null)
return result;
}
return null;
}
If you put this code block in your TreeModel class, the example you provided will return m2
Absolutely not, with a child node like that you can't get its parent. Simply because there isn't any reference to it.
To get the parent of the node, you have to either add the parent field or save the reference in somewhere else (by a variable or something).
EDIT
#Zulis If you search from the root node, you can definitely find the node you want. But as I said, with just the child node you can't do that.
But I think you should avoid searching because that would be slow

Recursive tree creation

I have a Node class in C# with the following properties:
public class Node
{
public int Id {get;set;}
public int? ParentId {get;set;}
public string Label {get;set;}
}
I have a TreeView control which provides the following method to create
a new node:
MyTreeView.CreateNode(key, label);
parent.Nodes.CreateNode(key, label);
If I want to add a new child node I need to use the second method otherwise the first one. Both returns an object of type TreeNode.
How would you create in C# a recursive function to populate the treeview considering that the root nodes have ParentId = null?
This is what I have done so far:
// create a list of root nodes
var roots = myList.Where(x => x.ParentId == null);
// send the roots to a recursive func
foreach(var root in roots)
{
AddNode(null,root,myList);
}
this is my recursive function:
private void AddNode(Node parent, Node current, IList<Node> items)
{
TreeNode treenode = null;
if(parent == null)
{
treenode = mytree.CreateNode(current.Id.ToString(), current.Label);
}else{
var parentnode = mytree.GetNode(parent.Id.ToString());
treenode = parentnode.Nodes.CreateNode(current.Id.ToString(), current.Label);
}
// call the recursion for the children
var children = items.Where(x => x.ParentId == current.Id);
foreach(var child in children)
{
AddNode(current, child, items);
}
}
If your tree view control is derived from System.Windows.Forms.TreeView you can replace
MyTreeView.CreateNode(key, label);
parent.Nodes.CreateNode(key, label);
with
MyTreeView.Nodes.Add(key, label);
parent.Nodes.Add(key, label);
So the call always goes to a Nodes collection which is of type TreeNodeCollection. Instead of your Node object you can now use the Nodes collection as parameter.
var roots = myList.Where(x => x.ParentId == null);
foreach (var root in roots)
{
AddNode(mytree.Nodes, root, myList);
}
private void AddNode(TreeNodeCollection nodes, Node current, IList<Node> items)
{
TreeNode treenode = nodes.Add(current.Id.ToString(), current.Label);
var children = items.Where(x => x.ParentId == current.Id);
foreach (var child in children)
{
AddNode(treenode.Nodes, child, items);
}
}
This has two benefits:
You don't need to lookup the parent each time.
You only have one call (TreeNodeCollection.Add).
However, if you can not access the TreeView.Nodes collection in the AddNode call for each root you will have to check for that at the top of the AddNode method.
var roots = myList.Where(x => x.ParentId == null);
foreach (var root in roots)
{
AddNode(null, root, myList);
}
private void AddNode(TreeNodeCollection nodes, Node current, IList<Node> items)
{
if (nodes == null)
{
nodes = myTree.Nodes;
}
...
}
Try this code:
var lookup = myList.ToLookup(n => n.ParentId.ToString());
Action<IEnumerable<TreeNode>> addChildren = null;
addChildren = tns =>
{
var query =
from tn in tns
from cn in lookup[tn.Name]
select tn.Nodes.CreateNode(cn.Id.ToString(), cn.Label);
var nodes = query.ToArray();
if (nodes.Length > 0)
{
addChildren(nodes);
}
};
addChildren(
lookup[null]
.Select(n =>
MyTreeView.CreateNode(n.Id.ToString(), n.Label)));
I couldn't fully test it, so you might need to change some of the code to make it work, but it should be fairly close.
I'd probably do something like this...
public class Node
{
public int Id { get; set; }
public int? ParentId { get; set; }
public string Label { get; set; }
public Node(int? parentId, int id, string label)
{
ParentId = parentId;
Id = id;
Label = label;
}
}
public class TreeNode : List<TreeNode>
{
public string Key { get; set; }
public string Label { get; set; }
public IEnumerable<TreeNode> Descendants
{
get
{
yield return this;
foreach (var child in this)
{
foreach (var descendant in child.Descendants)
{
yield return descendant;
}
}
}
}
public TreeNode(string key, string label)
{
Key = key;
Label = label;
}
public void CreateNode(int id, string label)
{
Add(new TreeNode(id.ToString(), label));
}
}
public class Tree
{
private TreeNode _root = new TreeNode(null, null);
public Tree(IEnumerable<Node> nodes)
{
nodes.ToList().ForEach(node => CreateNode(node.ParentId, node.Id, node.Label));
}
public void CreateNode(int? parentId, int id, string label)
{
if (parentId == null)
{
_root.CreateNode(id, label);
}
else
{
_root.Descendants.First(x => x.Key == parentId.ToString()).CreateNode(id, label);
}
}
public IEnumerable<TreeNode> Descendants => _root.Descendants;
}
Try this code:
Node{
Id, Label, List<Tree> Children
}
Tree GetTree(id){
var node=new Node();
node.Id=id;
node.Children=new List<Node>();
List<Node> children = db.Nodes.Where(x => x.ParentId==id);
foreach(child in children){
var childTree=GetTree(child.Id);
node.Children.Add(childTree);
}
return node;
}

Categories