I am using windows azure table storage. My problem is that the access of entities for a given user from table is taking long time. The code which i am using to access the table is as follows :
public CloudTableQuery<T> GetEntites(string username)
{
try
{
CloudTableQuery<T> entries =
(from e in ServiceContext.CreateQuery<T>(TableName)
where e.PartitionKey == username
select e).AsTableServiceQuery();
return entries;
}
catch (Exception)
{ return null; }
}
The total entities in the table are currently only 100 or so. For example : The query seems to take as long as 40 seconds to return 25 entities for a given user. Please suggest if there is any scope for improvement in the code for faster performance ?
The real problem here is you are returning queries instead of entities, and you are likely executing a query again for each returned entity (query). Define your entity with something like:
public class UserEntity
{
public string UserName { get; set; }
public string Name { get; set; }
public string Email { get; set; }
}
Then define your table storage entity with something like:
public class StoredUserEntity : TableServiceEntity
{
public StoredUserEntity (string username)
{
this.PartitionKey = username;
this.RowKey = Guid.NewGuid().ToString();
this.Timestamp = DateTime.Now;
}
public string Email { get; set; }
public string Name { get; set; }
}
Then your query should actually return a list of UserEntities:
public List<UserEntity> GetUserData(string username)
{
var q = _tableContext.CreateQuery<StoredUserEntity>(_tableName);
return
q.ToList().Select(x => new UserEntity {UserName = x.PartitionKey, Email = x.Email, Name = x.Name}).Where(x => x.UserName == username).ToList();
}
Try using Fiddler to see what's going on. It could be that you are experiencing a few retries that could slow things down. The API won't tell you that, but with Fiddler you can see exactly the requests being made, any responses (along with errors) and how many of them are being made.
Related
I'm creating a system that draws routes using Bing Maps and ASP.Net. I want to store the postcodes for the stops on the route in a SQL Database and pass them over to the view using MVC.
However, querying the database doesn't return any results, even though the Database is popualted. I've used Debug.WriteLine in various parts of the code and deduced that though it's connecting to the database and querying the right table, when I initialise the DBContext in my controller, using ToList.Count on it shows 0.
My code is as follows:
Controller - MapController.cs
private StopsDBContext db = new StopsDBContext();
public ActionResult Index()
{
Debug.WriteLine("***************************************************************************************");
Debug.WriteLine("--->" + db.Stops.ToList().Count + "<---"); // this shows 0
Debug.WriteLine("***************************************************************************************");
var Stops = from e in db.Stops
select e;
List<Stop> Model = Stops.ToList();
return View(Model);
}
Model - Stop.cs
public class Stop
{
public int ID { get; set; }
public string Name { get; set; }
public string Postcode { get; set; }
public Stop(int iD, string name, string postcode)
{
ID = iD;
Name = name;
Postcode = postcode;
}
public Stop() {}
}
public class StopLists
{
public IEnumerable<Stop> StopList { get; set; }
}
public class StopsDBContext : DbContext
{
public StopsDBContext()
{
Database.Log = s => Debug.WriteLine(s); //this shows the sql queries in the console
}
public DbSet<Stop> Stops { get; set; }
}
I've passed hardcoded data into the view before doing this so I know that part's right, and since it's running the query it must be connecting to the Database, so I'm not sure what I'm missing.
Thanks in advance
EDIT: Updated classes following advice from Dai
I think that you should use the constructor like this way :
public StopsDBContext() : base("name=Connectionname")
I've solved the error now. Thanks to everyone that replied for your advice.
Turns out my model and my table had different fields and I didn't realise, so the DBContext was searching for fields that didn't exist. Whoops.
In my Xamarin application I have a local db set up using SQLite. I have followed the microsoft to-do list example when setting this up (https://learn.microsoft.com/en-gb/xamarin/xamarin-forms/data-cloud/data/databases) however I have hit an issue.
I have a model called "Animal" which is saved into my database and what I am trying to query the fields of:
public int ID { get; set; }
public string Name {get; set; }
public string Species { get; set; }
public string Markings { get; set; }
public string Diet { get; set; }
Within my database query file I am trying to perform a query that will get the animal based upon it's name which is given by the user and then from this it will return its markings/diet/etc...
This is what I have so far but it is erroring and I am really not sure on how to fix it:
public async Task<string> getanimaldiet(string name)
{
Animal a = Database.Table<Animal>().Where(x => x.Name == name).FirstOrDefaultAsync();
return a.Diet;
}
Errors:
The name 'diet' does not exist in the current context & Cannot implicitly convert type 'System.Threading.Tasks.Task<animalService.Animal>' to 'animalService.Animal'
Any help is much appreciated.
Since you have async api you need to add await to your code:
Animal a = await Database.Table<Animal>().Where(x => x.Name == name)
.FirstOrDefaultAsync();
return a.Diet;
You can make it sync:
public string getanimaldiet(string name)
{
Animal a = Database.Table<Animal>().Where(x => x.Name == name)
.FirstOrDefault();
return a.Diet;
)
In addition to my earlier question on entity framework. My purpose is to encrypt the field before saving to DB and Decrypt the fields before reading from DB.I have class (User.cs) generated from table User(having UserName property(to be encrypted) and I created a separate class SecureUser with UserName property that isencrypted/decrypted as suggested earlier. But I am not sure how to map this new SecureUser class to DB and not the previous POCO class User. My Linq queries fail when I replace UserName with UserName from SecureUser class. I tried doing the same thing with partial classes , same thing happens.
Any suggestions will be greatly appreciated !
[Table("Users")]
public class User
{
#region database table column mapped fields
[Key]
[Required]
public Int32 UserID { set; get; }
[Required]
[MaxLength(50)]
public String UserName { set; get; }
[Required]
public Int32 CustID { set; get; }
//created the separate class as
public class SecureUser // UserViewModel
{
//
private readonly User _user;
public SecureUser(User user)
{
_user = user;
}
public string UserName
{
get { return Decrypt(_user.UserName); }
set { _user.UserName = Encrypt(value); }
}
}
I am a big fan of any solution that depends on Attributes.
Imagine you have an Entity class with one or more Properties you want to store Encrypted in the DB.Just add [Encrypted] attribute.
[Encrypted]
public string EncryptedProperty { get; set; }
Now create a custom DbContext that does Encryption/Decryption on the fly for you:
public class MyDB : IdentityDbContext<User>
{
//DBSet properties go here
public MyDB()
{
((IObjectContextAdapter)this).ObjectContext.ObjectMaterialized += new ObjectMaterializedEventHandler(ObjectMaterialized);
}
#region Encryption
public override int SaveChanges()
{
var contextAdapter = ((IObjectContextAdapter)this);
contextAdapter.ObjectContext.DetectChanges(); //force this. Sometimes entity state needs a handle jiggle
var pendingEntities = contextAdapter.ObjectContext.ObjectStateManager
.GetObjectStateEntries(EntityState.Added | EntityState.Modified)
.Where(en => !en.IsRelationship).ToList();
foreach (var entry in pendingEntities) //Encrypt all pending changes
EncryptEntity(entry.Entity);
int result = base.SaveChanges();
foreach (var entry in pendingEntities) //Decrypt updated entities for continued use
DecryptEntity(entry.Entity);
return result;
}
public override async Task<int> SaveChangesAsync(System.Threading.CancellationToken cancellationToken)
{
var contextAdapter = ((IObjectContextAdapter)this);
contextAdapter.ObjectContext.DetectChanges(); //force this. Sometimes entity state needs a handle jiggle
var pendingEntities = contextAdapter.ObjectContext.ObjectStateManager
.GetObjectStateEntries(EntityState.Added | EntityState.Modified)
.Where(en => !en.IsRelationship).ToList();
foreach (var entry in pendingEntities) //Encrypt all pending changes
EncryptEntity(entry.Entity);
var result = await base.SaveChangesAsync(cancellationToken);
foreach (var entry in pendingEntities) //Decrypt updated entities for continued use
DecryptEntity(entry.Entity);
return result;
}
void ObjectMaterialized(object sender, ObjectMaterializedEventArgs e)
{
DecryptEntity(e.Entity);
}
private void EncryptEntity(object entity)
{
//Get all the properties that are encryptable and encrypt them
var encryptedProperties = entity.GetType().GetProperties()
.Where(p => p.GetCustomAttributes(typeof(Encrypted), true).Any(a => p.PropertyType == typeof(String)));
foreach (var property in encryptedProperties)
{
string value = property.GetValue(entity) as string;
if (!String.IsNullOrEmpty(value))
{
string encryptedValue = EncryptionService.Encrypt(value);
property.SetValue(entity, encryptedValue);
}
}
}
private void DecryptEntity(object entity)
{
//Get all the properties that are encryptable and decyrpt them
var encryptedProperties = entity.GetType().GetProperties()
.Where(p => p.GetCustomAttributes(typeof(Encrypted), true).Any(a => p.PropertyType == typeof(String)));
foreach (var property in encryptedProperties)
{
string encryptedValue = property.GetValue(entity) as string;
if (!String.IsNullOrEmpty(encryptedValue))
{
string value = EncryptionService.Decrypt(encryptedValue);
this.Entry(entity).Property(property.Name).OriginalValue = value;
this.Entry(entity).Property(property.Name).IsModified = false;
}
}
}
#endregion Encryption
}
Then add a class
class Encrypted : Attribute
{
}
Source: https://gist.github.com/albertbori/e95860644e69c1572441
You can't use SecureUser's UserName property in a LINQ to Entities query because it's calling the Decrypt method, which LINQ has no idea on how to translate that into SQL. You should probably just create a [NotMapped] property on your regular User object, and that should do it. Like so:
[Table("Users")]
public class User
{
#region database table column mapped fields
[Key]
[Required]
public Int32 UserID { set; get; }
[Required]
[MaxLength(50)]
public String UserName { set; get; }
[Required]
public Int32 CustID { set; get; }
[NotMapped]
public string DecryptedUserName
{
get { return Decrypt(this.UserName); }
set { this.UserName = Encrypt(value); }
}
}
If that doesn't work, you'll have to do the decryption after the objects have come back from the database inside of a LINQ to Objects query. Something like this:
var users = db.Users.Where(u => u.CustID == someID); //LINQ to Entities
var decryptedUserNames = users.ToList().Select(u => Decrypt(u.UserName)); //LINQ to Objects
On adding the user object to the dbcontext, you should set UserName=Encrypt(model.UserName) where model is the object returned by the user for adding it
// before adding the model to the dbcontext
model.UserName=Encrypt(mode.UserName);
db.Users.Add(model);
db.SaveChanges();
[Table("Users")]
public class User
{
#region database table column mapped fields
[Key]
[Required]
public Int32 UserID { set; get; }
[Required]
[MaxLength(50)]
public String UserName { set; get; } // this field will be encrypted field
[Required]
public Int32 CustID { set; get; }
[NotMapped]
public string DecryptedUserName
{
get { return Decrypt(UserName); } // this you can display it to the user
}
}
to use the NotMapped, you should add this line in the using statements
using System.ComponentModel.DataAnnotations.Schema;
hope this will help you
You can also use the free Crypteron CipherDb library that works transparently with encrypted data. There is also a feature to allow for searchable encryption if that's something you'll be using in your LINQ queries.
You can annotate the data model with [Secure] or name a property to something like Secure_SocialSecurityNumber (the Secure_ is the key part) and CipherDb automatically performs data encryption, tamper protection, secure key storage, secure key distribution, caching, key roll overs, ACLs and more. Alternatively, with the CipherObject adaptor you can just use myObject.Seal() to encrypt the object.
You can find the sample apps on GitHub at https://github.com/crypteron/crypteron-sample-apps. You can also use Crypteron to protect streams, files, objects, message queues, noSQL etc.
Disclaimer: I work there and we do have a free community edition which anyone can use.
I have a piece of code that should retrieve a List of User objects.
public List<User> GetUsersBySessions(string[] sessionStrs, string ip)
{
if (sessionStrs == null || string.IsNullOrEmpty(ip))
return new List<User>();
using (var ctx = new DataContext())
{
var ret = ctx.Sessions.Include("User").Where(s => sessionStrs.Contains(s.ID) && s.IP.Equals(ip)).Select(s => s.User).ToList();
return ret;
}
}
The arguments sessionStrs and ip are properly passed into the method. However, I'm getting the following error:
(source: imgbomb.com)
How could this type of error be caused when I'm not using any .First() or .Single()? I'm just trying to get all items in a table that fit into an array return them as a List. I've never seen such a problem before.
The following line of code even causes an error:
var ret = ctx.Sessions.ToList();
Here is my DataContext:
public class DataContext : DbContext
{
public GenericDataContext()
: base(CONNECTION_STRING) //My CONNECTION_STRING is defined somewhere else, but I'm going to hide that for security's sake.
{
}
public DbSet<Password> Passwords { get; set; }
public DbSet<Session> Sessions { get; set; }
public DbSet<User> Users { get; set; }
}
And here is my Session model:
[Table("tbl_Sessions")]
public class Session
{
[Column("SessionID")]
[MaxLength(24)]
[Required]
[Key]
public string ID { get; set; }
[Column("UserID")]
[Required]
public int UserID { get; set; }
[Column("IP")]
[MaxLength(24)]
[Required]
public string IP { get; set; }
[ForeignKey("UserID")]
public virtual User User { get; set; }
}
NOTE
Both Classes are properly namespaced and have their proper using statements, as well.
The actual answer to this question had to do with the fact that one of my models had two Keys, each of which was a string. When I changed said model to have an int as it's Key, all worked well.
May be sessionStrs is null:
public List<User> GetUsersBySessions(string[] sessionStrs, string ip)
{
using (var ctx = new DataContext())
{
var ret = ctx.Sessions.Include("User");
if(sessionStrs!=null && sessionStrs.Any())
ret =ret.Where(s => sessionStrs.Contains(s.ID));
if(!string.IsNullOrEmpty(ip))
ret =ret.Where(s => s.IP.Equals(ip));
return ret.Any()?ret.Select(s => s.User).ToList():NULL;
}
}
Try to check if you have elements before selecting:
var ret = ctx.Sessions.Where(s => sessionStrs.Contains(s.ID) && s.IP.Equals(ip));
if (ret.Any())
return ret.Select(s => s.User).ToList();
else
return null; // or new List<User>();
Google led me here, so I'll throw my experience out there for anyone having a similar issue.
My project was using EF fluent configuration to query an existing database. I couldn't figure out why I was receiving any data back despite being able to query the table with other tools.
After banging my head against the wall, I finally discovered that I'd mislabeled the column type like so:
Property(x => x.DateCreated)
.HasColumnName("datetiem") // strings are the devil
Make sure you're using FirstOrDefault instead of First if you're extracting something from a list.
I have a table called Activity (Mobile service), and i add a query in reading:
Azure Script:
function read(query, user, request)
{
var param = request.parameters.UserLocation;
if(param)
{
var sql = "Select TOP 10 [NewsItemUrl], count(1) as CounterNews FROM [MobileServiceExtra].[ACTIVITY] WHERE [UserLocation] = ? GROUP BY [NewsItemUrl] ORDER BY CounterNews Desc";
mssql.query(sql,param, {success: function(results) {request.respond(statusCodes.OK, results);}});
}
//request.execute();
}
Client side:
public class ACTIVITY
{
public int Id { get; set; }
[JsonProperty(PropertyName = "UserLocation")]
public string _UserLocation { get; set; }
[JsonProperty(PropertyName = "NewsItemUrl")]
public string _NewsItemUrl { get; set; }
[JsonProperty(PropertyName = "NewsItemTitle")]
public string _NewsItemTitle { get; set; }
[JsonProperty(PropertyName = "NewsItemPublisher")]
public string _NewsItemPublisher { get; set; }
}
If I do the query in sql, I get 2 columns CounterNews and NewsItemUrl and where the Last is the number of times to repeat the url. However, I dont know how to get the data in column "CounterNews", i mean, when i want to get the query, i get to do with the Activity table (class) and obviously returns me the data correctly, but only NewsItemUrl column and the other fields are empty.
Client side:
private MobileServiceCollection<ACTIVITY, ACTIVITY> TopReadCollectionActivity;
private IMobileServiceTable<ACTIVITY> ACTIVITYTable = App.MobileService.GetTable<ACTIVITY>();
private async void LoadTop10()
{
var dict = new Dictionary<string, string>
{
{ "UserLocation", "United States" },
};
try
{
TopReadCollectionActivity = await ACTIVITYTable.WithParameters(dict).ToCollectionAsync();
}
catch (Exception ex)
{
string err = ex.Message;
}
}
if i create a class (table too)"Top10", azure give a error, because the table doen't exist. and
not want to create a new table.
how to get the query only two fields CounterNews and NewsItemUrl?
Perhaps you should just query the table directly using the OData support. This way your not confined to the structure of your table. Take a look at this for reference.
http://msdn.microsoft.com/en-us/library/windowsazure/jj677199.aspx