We Keep Coding

ElasticSearch: How to fetch data from fields using ElasticClient from Nest 7.11?

I am using ElasticClient.Search function to get the values of the fields.
The issue is :
The code that i make below make the mapping correctly but for searching it returns null values of the fields that was mapped before.
Main.cs
using Nest;
using System;
using System.Linq;
using System.Threading;
namespace DataAccessConsole
{
class Program
{
public static Uri node;
public static ConnectionSettings settings;
public static ElasticClient client;
static void Main(string[] args)
{
{
node = new Uri("http://localhost:9200");
settings = new ConnectionSettings(node).DefaultIndex("getallcommissionspermanentes");
settings.DefaultFieldNameInferrer(p => p);
client = new ElasticClient(settings);
var indexSettings = new IndexSettings();
indexSettings.NumberOfReplicas = 1;
indexSettings.NumberOfShards = 1;
client.Indices.Create("getallcommissionspermanentes", index => index
.Map<GetAllCommissionsPermanentes>(
x => x
.AutoMap<GetAllCommissionsPermanentes>()
));
client.Search<GetAllCommissionsPermanentes>(s => s
.AllIndices()
);
}
}
GetAllCommissionsPermanentes.cs
the table is located in an edmx model of Entityframework and Data came from SQL SERVER Database
public partial class GetAllCommissionsPermanentes
{
public int ID { get; set; }
public string NomAr { get; set; }
public string NomFr { get; set; }
}
if you need more informations just make a comment below.
Thanks

Code is correct but '.All Indices ()' searches in all indexes, results that do not match the model are coming. This code will return more accurate results;
client.Search<GetAllCommissionsPermanentes>(s => s.Index("getallcommissionspermanentes");

EF Core - Add new tables to database During Runtime

I have an asp.net core project which needs to be able to support plugins at runtime, and as a consequence, I need to generate database tables based on what has been plugged in. The plugins are each divided in separate projects and they have have their own DbContext class. The plugins to be used are not known during compile-time, only at runtime.
Now in EF Core I thought that there would be a method like "UpdateDatabase" where you can just add tables to the existing database, but I was wrong. Is there a way to accomplish this? I was able to generate a separate database for each of the plugins, but that wasn't quite what I had in mind..I needed all tables in one database.
Here's the code for the "HRContext" plugin:
using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.EntityFrameworkCore;
using Plugins.HR.Models.Entities;
namespace Plugins.HR.Contexts
{
public class HrContext : DbContext
{
public HrContext()
{
}
public HrContext(DbContextOptions<HrContext> contextOptions) : base(contextOptions)
{
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.HasDefaultSchema("HR");
base.OnModelCreating(modelBuilder);
}
public DbSet<Address> Address { get; set; }
public DbSet<Attendance> Attendance { get; set; }
public DbSet<Department> Departments { get; set; }
public DbSet<Employee> Employees { get; set; }
public DbSet<JobTitle> JobTitles { get; set; }
}
}
Here's another piece of code for the "CoreContext" plugin:
using System;
using System.Collections.Generic;
using System.Text;
using Core.Data.Models;
using Microsoft.EntityFrameworkCore;
namespace Core.Data.Contexts
{
public class CoreContext : DbContext
{
public CoreContext()
{
}
public CoreContext(DbContextOptions<CoreContext> contextOptions) : base(contextOptions)
{
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.HasDefaultSchema("Core");
base.OnModelCreating(modelBuilder);
}
public DbSet<Test> Tests { get; set; }
}
}
My ConfigureServices method in Startup.cs:
public void ConfigureServices(IServiceCollection services)
{
services.AddDbContext<CoreContext>(options => options.UseSqlServer("Data source = localhost; initial catalog = Company.Core; integrated security = true;"))
.AddDbContext<HrContext>(options => options.UseSqlServer("Data source = localhost; initial catalog = Company.HR; integrated security = true;"));
// Add framework services.
services.AddMvc();
}
If I try to change the connection string to be the same, sooner or later I will get an error that says that the table for one plugin does not exist. I tried "EnsureCreated" but that didn't work too.

I had the same issue. See my solution on GitHub a few days ago, here: EF Core Issue #9238
What you need is something like the following:
// Using an interface, so that we can swap out the implementation to support PG or MySQL, etc if we wish...
public interface IEntityFrameworkHelper
{
void EnsureTables<TContext>(TContext context)
where TContext : DbContext;
}
// Default implementation (SQL Server)
public class SqlEntityFrameworkHelper : IEntityFrameworkHelper
{
public void EnsureTables<TContext>(TContext context)
where TContext : DbContext
{
string script = context.Database.GenerateCreateScript(); // See issue #2943 for this extension method
if (!string.IsNullOrEmpty(script))
{
try
{
var connection = context.Database.GetDbConnection();
bool isConnectionClosed = connection.State == ConnectionState.Closed;
if (isConnectionClosed)
{
connection.Open();
}
var existingTableNames = new List<string>();
using (var command = connection.CreateCommand())
{
command.CommandText = "SELECT table_name from INFORMATION_SCHEMA.TABLES WHERE table_type = 'base table'";
using (var reader = command.ExecuteReader())
{
while (reader.Read())
{
existingTableNames.Add(reader.GetString(0).ToLowerInvariant());
}
}
}
var split = script.Split(new[] { "CREATE TABLE " }, StringSplitOptions.RemoveEmptyEntries);
foreach (string sql in split)
{
var tableName = sql.Substring(0, sql.IndexOf("(", StringComparison.OrdinalIgnoreCase));
tableName = tableName.Split('.').Last();
tableName = tableName.Trim().TrimStart('[').TrimEnd(']').ToLowerInvariant();
if (existingTableNames.Contains(tableName))
{
continue;
}
try
{
using (var createCommand = connection.CreateCommand())
{
createCommand.CommandText = "CREATE TABLE " + sql.Substring(0, sql.LastIndexOf(";"));
createCommand.ExecuteNonQuery();
}
}
catch (Exception)
{
// Ignore
}
}
if (isConnectionClosed)
{
connection.Close();
}
}
catch (Exception)
{
// Ignore
}
}
}
}
Then at the end of Startup.Configure(), I resolve an IEntityFrameworkHelper instance and use that with an instance of DbContext to call EnsureTables().
One issue is I need to still account for the parts of the script which are not CREATE TABLE statements. For example, the CREATE INDEX statements.
I requested they give us a clean solution, for example: add a CreateTable<TEntity>() method to IRelationalDatabaseCreator. Not holding my breath for that though...
EDIT
I forgot to post the code for GenerateCreateScript(). See below:
using System.Text;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage;
public static class DatabaseFacadeExtensions
{
public static string GenerateCreateScript(this DatabaseFacade database)
{
var model = database.GetService<IModel>();
var migrationsModelDiffer = database.GetService<IMigrationsModelDiffer>();
var migrationsSqlGenerator = database.GetService<IMigrationsSqlGenerator>();
var sqlGenerationHelper = database.GetService<ISqlGenerationHelper>();
var operations = migrationsModelDiffer.GetDifferences(null, model);
var commands = migrationsSqlGenerator.Generate(operations, model);
var stringBuilder = new StringBuilder();
foreach (var command in commands)
{
stringBuilder
.Append(command.CommandText)
.AppendLine(sqlGenerationHelper.BatchTerminator);
}
return stringBuilder.ToString();
}
}
It's based on the code found here: EF Core Issue #2943

Entity Framework 6 set connection string in code

I have a dll that uses the Entity Framework 6 to do some database operations. I'm using a database first approach.
The model and everything concerning the Entity Framework, like the connection string in the App.config, were created via the wizzard in Visual Studio.
So I compiled the dll and put it together with the corresponding .config in the folder where the application using the dll expects it.
Everything works fine until I get to the point where an actual database call is made. There I get the error:
Cannot find connection string for MyDatabaseEntity
The automatically generated connectionstring is, as I said, in the config file of the dll. I cannot change the App.config of the application.
But the application hands over an object that has all the information I need to build the connection string myself.
So I'm looking for a way to set the connection string in the code without relying on a config file.
All the tutorials I find for a database first approach use this method though.
I found a post here that says to simply give the connection string as a parameter when creating the Object like
MyDatabaseEntities = new MyDatabaseEntities(dbConnect);
but ´MyDatabaseEntities´ doesn't have a constructor that takes any parameters
public partial class MyDatabaseEntities : DbContext
{
public MyDatabaseEntities()
: base("name=MyDatabaseEntities")
{
}
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
throw new UnintentionalCodeFirstException();
}
public virtual DbSet<MyTable> MyTable { get; set; }
}

How about:
public partial class MyDatabaseEntities : DbContext
{
public MyDatabaseEntities(string connectionString)
: base(connectionString)
{
}
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
throw new UnintentionalCodeFirstException();
}
public virtual DbSet<MyTable> MyTable { get; set; }
}
Then initialize your database like you did before:
string myConnectionString = "...";
MyDatabaseEntities = new MyDatabaseEntities(myConnectionString);

I had the similar issue. My Edmx and App.Config was in a different project. My startup project was different, had 3 different connection strings, we need to choose one on the fly depending on the environment. So couldn't use a fixed connection string. I created a partial class overload of the Context.cs using the same namespace. Following was my default Context.cs;
namespace CW.Repository.DBModel
{
public partial class CWEntities : DbContext
{
public CWEntities()
: base("name=CWEntities")
{
}
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
throw new UnintentionalCodeFirstException();
}
...
...
}
}
My partial class overload;
namespace CW.Repository.DBModel
{
public partial class CWEntities : DbContext
{
public CWEntities(string ConnectionString)
: base(ConnectionString)
{
}
}
}
Lastly, as my connection strings were not for EF, I converted them to a EF connection string.
public static string GetEntityConnectionString(string connectionString)
{
var entityBuilder = new EntityConnectionStringBuilder();
// WARNING
// Check app config and set the appropriate DBModel
entityBuilder.Provider = "System.Data.SqlClient";
entityBuilder.ProviderConnectionString = connectionString + ";MultipleActiveResultSets=True;App=EntityFramework;";
entityBuilder.Metadata = #"res://*/DBModel.CWDB.csdl|res://*/DBModel.CWDB.ssdl|res://*/DBModel.CWDB.msl";
return entityBuilder.ToString();
}
Lastly, the calling
var Entity = new CWEntities(CWUtilities.GetEntityConnectionString(ConnectionString));

I got this solution using below code, I can hardcode connection string using C# code without using config file.
public class SingleConnection
{
private SingleConnection() { }
private static SingleConnection _ConsString = null;
private String _String = null;
public static string ConString
{
get
{
if (_ConsString == null)
{
_ConsString = new SingleConnection { _String = SingleConnection.Connect() };
return _ConsString._String;
}
else
return _ConsString._String;
}
}
public static string Connect()
{
//Build an SQL connection string
SqlConnectionStringBuilder sqlString = new SqlConnectionStringBuilder()
{
DataSource = "SIPL35\\SQL2016".ToString(), // Server name
InitialCatalog = "Join8ShopDB", //Database
UserID = "Sa", //Username
Password = "Sa123!##", //Password
};
//Build an Entity Framework connection string
EntityConnectionStringBuilder entityString = new EntityConnectionStringBuilder()
{
Provider = "System.Data.SqlClient",
Metadata = "res://*/ShopModel.csdl|res://*/ShopModel.ssdl|res://*/ShopModel.msl",
ProviderConnectionString = #"data source=SIPL35\SQL2016;initial catalog=Join8ShopDB2;user id=Sa;password=Sa123!##;"// sqlString.ToString()
};
return entityString.ConnectionString;
}
and using DbContext using like this:
Join8ShopDBEntities dbContext = new Join8ShopDBEntities(SingleConnection.ConString);

Thanks a lot . I changed little for Code First EF6.
using System;
using System.Collections.Generic;
using System.Configuration;
using System.Data.Entity.Core.EntityClient;
using System.Data.SqlClient;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace Data
{
public class SingleConnection
{
private SingleConnection() { }
private static SingleConnection _ConsString = null;
private String _String = null;
public static string ConString
{
get
{
if (_ConsString == null)
{
_ConsString = new SingleConnection { _String = SingleConnection.Connect() };
return _ConsString._String;
}
else
return _ConsString._String;
}
}
public static string Connect()
{
string conString = ConfigurationManager.ConnectionStrings["YourConnectionStringsName"].ConnectionString;
if (conString.ToLower().StartsWith("metadata="))
{
System.Data.Entity.Core.EntityClient.EntityConnectionStringBuilder efBuilder = new System.Data.Entity.Core.EntityClient.EntityConnectionStringBuilder(conString);
conString = efBuilder.ProviderConnectionString;
}
SqlConnectionStringBuilder cns = new SqlConnectionStringBuilder(conString);
string dataSource = cns.DataSource;
SqlConnectionStringBuilder sqlString = new SqlConnectionStringBuilder()
{
DataSource = cns.DataSource, // Server name
InitialCatalog = cns.InitialCatalog, //Database
UserID = cns.UserID, //Username
Password = cns.Password, //Password,
MultipleActiveResultSets = true,
ApplicationName = "EntityFramework",
};
//Build an Entity Framework connection string
EntityConnectionStringBuilder entityString = new EntityConnectionStringBuilder()
{
Provider = "System.Data.SqlClient",
Metadata = "res://*",
ProviderConnectionString = sqlString.ToString()
};
return entityString.ConnectionString;
}
}
}

You can use singleton patter for it . For example
private YouurDBContext context;
public YouurDBContext Context
{
get
{
if (context==null)
{
context = new YouurDBContext();
}
return context;
}
set { context = value; }
}

Repeatedly creating and deleting databases in Entity Framework

When writing some unit tests for our application, I stumbled upon some weird behaviour in EF6 (tested with 6.1 and 6.1.2): apparently it is impossible to repeatedly create and delete databases (same name/same connection string) within the same application context.
Test setup:
public class A
{
public int Id { get; set; }
public string Name { get; set; }
}
class AMap : EntityTypeConfiguration<A>
{
public AMap()
{
HasKey(a => a.Id);
Property(a => a.Name).IsRequired().IsMaxLength().HasColumnName("Name");
Property(a => a.Id).HasColumnName("ID");
}
}
public class SomeContext : DbContext
{
public SomeContext(DbConnection connection, bool ownsConnection) : base(connection, ownsConnection)
{
}
public DbSet<A> As { get; set; }
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
modelBuilder.Configurations.Add(new AMap());
}
}
[TestFixture]
public class BasicTest
{
private readonly HashSet<string> m_databases = new HashSet<string>();
#region SetUp/TearDown
[TestFixtureSetUp]
public void SetUp()
{
System.Data.Entity.Database.SetInitializer(
new CreateDatabaseIfNotExists<SomeContext>());
}
[TestFixtureTearDown]
public void TearDown()
{
foreach (var database in m_databases)
{
if (!string.IsNullOrWhiteSpace(database))
DeleteDatabase(database);
}
}
#endregion
[Test]
public void RepeatedCreateDeleteSameName()
{
var dbName = Guid.NewGuid().ToString();
m_databases.Add(dbName);
for (int i = 0; i < 2; i++)
{
Assert.IsTrue(CreateDatabase(dbName), "failed to create database");
Assert.IsTrue(DeleteDatabase(dbName), "failed to delete database");
}
Console.WriteLine();
}
[Test]
public void RepeatedCreateDeleteDifferentName()
{
for (int i = 0; i < 2; i++)
{
var dbName = Guid.NewGuid().ToString();
if (m_databases.Add(dbName))
{
Assert.IsTrue(CreateDatabase(dbName), "failed to create database");
Assert.IsTrue(DeleteDatabase(dbName), "failed to delete database");
}
}
Console.WriteLine();
}
[Test]
public void RepeatedCreateDeleteReuseName()
{
var testDatabases = new HashSet<string>();
for (int i = 0; i < 3; i++)
{
var dbName = Guid.NewGuid().ToString();
if (m_databases.Add(dbName))
{
testDatabases.Add(dbName);
Assert.IsTrue(CreateDatabase(dbName), "failed to create database");
Assert.IsTrue(DeleteDatabase(dbName), "failed to delete database");
}
}
var repeatName = testDatabases.OrderBy(n => n).FirstOrDefault();
Assert.IsTrue(CreateDatabase(repeatName), "failed to create database");
Assert.IsTrue(DeleteDatabase(repeatName), "failed to delete database");
Console.WriteLine();
}
#region Helpers
private static bool CreateDatabase(string databaseName)
{
Console.Write("creating database '" + databaseName + "'...");
using (var connection = CreateConnection(CreateConnectionString(databaseName)))
{
using (var context = new SomeContext(connection, false))
{
var a = context.As.ToList(); // CompatibleWithModel must not be the first call
var result = context.Database.CompatibleWithModel(false);
Console.WriteLine(result ? "DONE" : "FAIL");
return result;
}
}
}
private static bool DeleteDatabase(string databaseName)
{
using (var connection = CreateConnection(CreateConnectionString(databaseName)))
{
if (System.Data.Entity.Database.Exists(connection))
{
Console.Write("deleting database '" + databaseName + "'...");
var result = System.Data.Entity.Database.Delete(connection);
Console.WriteLine(result ? "DONE" : "FAIL");
return result;
}
return true;
}
}
private static DbConnection CreateConnection(string connectionString)
{
return new SqlConnection(connectionString);
}
private static string CreateConnectionString(string databaseName)
{
var builder = new SqlConnectionStringBuilder
{
DataSource = "server",
InitialCatalog = databaseName,
IntegratedSecurity = false,
MultipleActiveResultSets = false,
PersistSecurityInfo = true,
UserID = "username",
Password = "password"
};
return builder.ConnectionString;
}
#endregion
}
RepeatedCreateDeleteDifferentName completes successfully, the other two fail. According to this, you cannot create a database with the same name, already used once before. When trying to create the database for the second time, the test (and application) throws a SqlException, noting a failed login. Is this a bug in Entity Framework or is this behaviour intentional (with what explanation)?
I tested this on a Ms SqlServer 2012 and Express 2014, not yet on Oracle.
By the way: EF seems to have a problem with CompatibleWithModel being the very first call to the database.
Update:
Submitted an issue on the EF bug tracker (link)

Database initializers only run once per context per AppDomain. So if you delete the database at some arbitrary point they aren't going to automatically re-run and recreate the database. You can use DbContext.Database.Initialize(force: true) to force the initializer to run again.

A few days ago I wrote integration tests that included DB access through EF6. For this, I had to create and drop a LocalDB database on each test case, and it worked for me.
I didn't use EF6 database initializer feature, but rather executed a DROP/CREATE DATABASE script, with the help of this post - I copied the example here:
using (var conn = new SqlConnection(#"Data Source=(LocalDb)\v11.0;Initial Catalog=Master;Integrated Security=True"))
{
conn.Open();
var cmd = new SqlCommand();
cmd.Connection = conn;
cmd.CommandText = string.Format(#"
IF EXISTS(SELECT * FROM sys.databases WHERE name='{0}')
BEGIN
ALTER DATABASE [{0}]
SET SINGLE_USER
WITH ROLLBACK IMMEDIATE
DROP DATABASE [{0}]
END
DECLARE #FILENAME AS VARCHAR(255)
SET #FILENAME = CONVERT(VARCHAR(255), SERVERPROPERTY('instancedefaultdatapath')) + '{0}';
EXEC ('CREATE DATABASE [{0}] ON PRIMARY
(NAME = [{0}],
FILENAME =''' + #FILENAME + ''',
SIZE = 25MB,
MAXSIZE = 50MB,
FILEGROWTH = 5MB )')",
databaseName);
cmd.ExecuteNonQuery();
}
The following code was responsible for creating database objects according to the model:
var script = objectContext.CreateDatabaseScript();
using ( var command = connection.CreateCommand() )
{
command.CommandType = CommandType.Text;
command.CommandText = script;
connection.Open();
command.ExecuteNonQuery();
}
There was no need to change database name between the tests.

When data is queried from database

According to MSDN documentation a LINQ query is not executed before iterated over in a foreach loop.
But when I try the following:
namespace MCSD487_AdoConnection
{
class Program
{
static void Main(string[] args)
{
DataSet dataSet = new DataSet();
dataSet.Locale = CultureInfo.InvariantCulture;
FillDataSet(dataSet);
DataTable folders = dataSet.Tables["Folder"];
IEnumerable<DataRow> folderQuery = folders.AsEnumerable();
IEnumerable<DataRow> aFolders = folderQuery.Where(f => f.Field<string>("Name")[0].ToString().ToLower() == "a");
// this is where I thought the SQL execution whould happen
foreach (DataRow row in aFolders)
{
Console.WriteLine("{0} was created on {1}", row.Field<string>("Name"), row.Field<DateTime>("DateTime"));
}
Console.ReadLine();
}
internal static void FillDataSet(DataSet dataSet)
{
try
{
string connectionString = ConfigurationManager.ConnectionStrings["conn"].ConnectionString;
SqlDataAdapter dataAdapter = new SqlDataAdapter("SELECT DateTime, Name FROM Folder", connectionString);
// Add table mappings.
dataAdapter.TableMappings.Add("Table", "Folder");
dataAdapter.Fill(dataSet);
// Fill the DataSet.
// This it where the actual SQL executes
dataAdapter.Fill(dataSet);
}
catch (SqlException ex)
{
Console.WriteLine("SQL exception occurred: " + ex.Message);
}
}
}
}
and I look at my SQL Server Profiler, I can see that the actual SQL call is performed when I call the dataAdapter.Fill(dataSet) in the FillDataSet method and not when iterating through rows.
My question is: How do I make LINQ perform the SQL execution on only the names starting with 'a' (without specifying that in the SQL commandText in the FillDataSet method)?
EDIT 2013-07-07 23:44:
I ended with the following solution, based on Evan Harpers answer:
using System;
using System.Data.SqlClient;
using System.Linq;
using System.Data.Linq;
using System.Data.Linq.Mapping;
namespace MCSD487_AdoConnection
{
[Table(Name = "Folder")]
public class Folder
{
private int _Id;
[Column(IsPrimaryKey = true, Storage = "_Id")]
public int Id
{
get { return _Id; }
set { _Id = value; }
}
private DateTime _DateTime;
[Column(Storage = "_DateTime")]
public DateTime DateTime
{
get { return _DateTime; }
set { _DateTime = value; }
}
private string _Name;
[Column(Storage = "_Name")]
public string Name
{
get { return _Name; }
set { _Name = value; }
}
}
class Program
{
static void Main(string[] args)
{
DataContext db = new DataContext(new SqlConnection(#"Data Source=OLF\OLF;Initial Catalog=DirStructure;Integrated Security=True"));
Table<Folder> folders = db.GetTable<Folder>();
IQueryable<Folder> folderQuery = from folder in folders
where folder.Name[0].ToString().ToLower() == "a"
select folder;
foreach (Folder folder in folderQuery)
{
Console.WriteLine("{0} was created on {1}", folder.Name, folder.DateTime);
}
Console.ReadLine();
}
}
}
Thanks for guiding me in the right direction.

This is exactly the expected behavior. You are using ADO.NET to get the data into a DataTable, and then querying against the DataTable – not the underlying database – in line with ADO.NET's disconnected architecture.
If you want to see your LINQ queries turned into optimized database calls just-in-time, use something like LINQ to SQL.

With DataSet you can'T do it. The only way is to fill the DataSet/Table adapter with the limited results (the starts with stuff have to be written in SQL).