I developed a website using Docker for its backend side. There are some components such as Selenium, Chrome Driver and MySQL which are integrated in docker-compose file. The problem is that when I move my project to VPS server (CentOS), docker compose could not be created properly and returns the following error:
web_1 | crit: Microsoft.AspNetCore.Hosting.Diagnostics[6]
web_1 | Application startup exception
web_1 | OpenQA.Selenium.WebDriverException: An unknown exception was encountered sending an HTTP request to the remote WebDriver server for URL http://selenium-hub:4444/wd/hub/session. The exception message was: Connection refused (selenium-hub:4444)
web_1 | ---> System.Net.Http.HttpRequestException: Connection refused (selenium-hub:4444)
web_1 | ---> System.Net.Sockets.SocketException (111): Connection refused
web_1 | at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
web_1 | at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
web_1 | at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|283_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
The interesting point is that containers are created and run flawlessly in my local host, but when I run them on VPS Server I am faced with this error.
This is my docker-compose file:
version: "3"
volumes:
datafiles:
services:
database:
image: mysql:8.0.28
depends_on:
- selenium-hub
ports:
- "3307:3306"
volumes:
- datafiles:/var/lib/mysql
- "~/sql-scripts/setup.sql:/docker-entrypoint-initdb.d/1.sql"
restart: always
environment:
MYSQL_ROOT_PASSWORD: *****
MYSQL_USER: sa
MYSQL_PASSWORD: ******
MYSQL_DATABASE: Pan
chrome:
image: selenium/node-chrome:4.3.0-20220706
shm_size: 2gb
depends_on:
- selenium-hub
environment:
- SE_EVENT_BUS_HOST=selenium-hub
- SE_EVENT_BUS_PUBLISH_PORT=4442
- SE_EVENT_BUS_SUBSCRIBE_PORT=4443
selenium-hub:
image: selenium/hub:4.3.0-20220706
container_name: selenium-hub
ports:
- "4442:4442"
- "4443:4443"
- "4444:4444"
web:
build: .
ports:
- "8080:80"
depends_on:
- database
- selenium-hub
- chrome
restart: always
And this is a piece of code I create my driver:
ChromeOptions options = new ChromeOptions();
options.AddArgument("no-sandbox");
options.AddArgument("headless");
driver = new RemoteWebDriver(new Uri("http://selenium-hub:4444/wd/hub"), options);
The way comes to my mind is that either I should modify my docker-compose file or give some permissions in VPS server to Docker.
The Issue:
Perform the following steps to recreate the issue:
cd C:\Temp
New-Item -ItemType Directory -Name init-db-issue
cd .\init-db-issue
# Download ASP.NET Core sample projects
Invoke-WebRequest -Uri https://codeload.github.com/dotnet/AspNetCore.Docs/zip/main -OutFile main.zip
# Expanding takes a while
Expand-Archive -Path .\main.zip
cd .\main\AspNetCore.Docs-main\aspnetcore\security\authorization\secure-data\samples\final3
dotnet user-secrets set SeedUserPW Secret123!
# Drop the database in case it already exists
sqlcmd -S "(localdb)\MSSQLLocalDB" -Q "DROP DATABASE [aspnet-ContactManager-02]"
At this point, do not load the project in vs or vscode. Simply run the project from a PowerShell prompt:
dotnet run
Here's what I get on my system:
PS C:\Temp\init-db-issue\main\AspNetCore.Docs-main\aspnetcore\security\authorization\secure-data\samples\final3> dotnet run
C:\Program Files\dotnet\sdk\5.0.200\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.EolTargetFrameworks.targets(28,5): warning NETSDK1138: The target framework 'netcoreapp3.0' is out of support and will not receive security updates in the future. Please refer to https://aka.ms/dotnet-core-support for more information about the support policy. [C:\Temp\init-db-issue\main\AspNetCore.Docs-main\aspnetcore\security\authorization\secure-data\samples\final3\ContactManager.csproj]
C:\Program Files\dotnet\sdk\5.0.200\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.EolTargetFrameworks.targets(28,5): warning NETSDK1138: The target framework 'netcoreapp3.0' is out of support and will not receive security updates in the future. Please refer to https://aka.ms/dotnet-core-support for more information about the support policy. [C:\Temp\init-db-issue\main\AspNetCore.Docs-main\aspnetcore\security\authorization\secure-data\samples\final3\ContactManager.csproj]
C:\Program Files\dotnet\sdk\5.0.200\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.EolTargetFrameworks.targets(28,5): warning NETSDK1138: The target framework 'netcoreapp3.0' is out of support and will not receive security updates in the future. Please refer to https://aka.ms/dotnet-core-support for more information about the support policy. [C:\Temp\init-db-issue\main\AspNetCore.Docs-main\aspnetcore\security\authorization\secure-data\samples\final3\ContactManager.csproj]
fail: ContactManager.Program[0]
An error occurred seeding the DB.
System.AggregateException: One or more errors occurred. (Value cannot be null. (Parameter 'password'))
---> System.ArgumentNullException: Value cannot be null. (Parameter 'password')
at Microsoft.AspNetCore.Identity.UserManager`1.CreateAsync(TUser user, String password)
at ContactManager.Data.SeedData.EnsureUser(IServiceProvider serviceProvider, String testUserPw, String UserName) in C:\Temp\init-db-issue\main\AspNetCore.Docs-main\aspnetcore\security\authorization\secure-data\samples\final3\Data\SeedData.cs:line 49
at ContactManager.Data.SeedData.Initialize(IServiceProvider serviceProvider, String testUserPw) in C:\Temp\init-db-issue\main\AspNetCore.Docs-main\aspnetcore\security\authorization\secure-data\samples\final3\Data\SeedData.cs:line 26
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at System.Threading.Tasks.Task.Wait()
at ContactManager.Program.Main(String[] args) in C:\Temp\init-db-issue\main\AspNetCore.Docs-main\aspnetcore\security\authorization\secure-data\samples\final3\Program.cs:line 35
info: Microsoft.Hosting.Lifetime[0]
Now listening on: http://localhost:5000
info: Microsoft.Hosting.Lifetime[0]
Now listening on: https://localhost:5001
info: Microsoft.Hosting.Lifetime[0]
Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
Hosting environment: Production
info: Microsoft.Hosting.Lifetime[0]
Content root path: C:\Temp\init-db-issue\main\AspNetCore.Docs-main\aspnetcore\security\authorization\secure-data\samples\final3
Note the following part:
fail: ContactManager.Program[0]
An error occurred seeding the DB.
System.AggregateException: One or more errors occurred. (Value cannot be null. (Parameter 'password'))
Now, load the project in vs but do not run it from there:
Start-Process .\ContactManager.csproj
Run the app from the PowerShell prompt:
dotnet run
Now I don't get the message An error occurred seeding the DB and the DB is indeed seeded.
Question:
Why does the issue go away when we simply load the project into Visual Studio? I would expect the project to work fine when running it via dotnet run without needing to also run Visual Studio. Is this a bug? Or is it due to some aspect of how the project is setup?
The project can be viewed here.
This project is the example used in the following tutorial:
Create an ASP.NET Core web app with user data protected by authorization
The issue is that Properties/launchSettings.json does not exist.
See this comment:
https://github.com/dotnet/AspNetCore.Docs/issues/21668#issuecomment-795543707
Here's a minimal test case which demonstrates the issue:
cd C:\Temp
mkdir init-db-issue-new-app
cd init-db-issue-new-app
dotnet new webapp -o ContactManager -au Individual -uld
cd .\ContactManager
dotnet user-secrets set SeedUserPW abc
#"
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using System;
namespace ContactManager
{
public class Program
{
public static void Main(string[] args)
{
var host = CreateHostBuilder(args).Build();
var config = host.Services.GetRequiredService<IConfiguration>();
var testUserPw = config["SeedUserPW"];
Console.WriteLine("{0}", testUserPw);
}
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseStartup<Startup>();
});
}
}
"# | Set-Content .\Program.cs
dotnet run # abc is displayed
mv .\Properties\launchSettings.json .\Properties\_launchSettings.json
dotnet run # abc is not displayed
When the application is published on windows it works perfectly. I am not using SQLite, I´m using SqlServer.
Error log from azure:
2020-11-27T17:36:08.879028391Z Documentation: http://aka.ms/webapp-linux
2020-11-27T17:36:08.879035191Z Dotnet quickstart: https://aka.ms/dotnet-qs
2020-11-27T17:36:08.879041791Z ASP .NETCore Version: 3.1.8
2020-11-27T17:36:08.879048191Z Note: Any data outside '/home' is not persisted
2020-11-27T17:36:09.077744493Z Running oryx create-script -appPath /home/site/wwwroot -output /opt/startup/startup.sh -defaultAppFilePath /defaulthome/hostingstart/hostingstart.dll -bindPort 8080 -userStartupCommand ''
2020-11-27T17:36:09.582281718Z Cound not find build manifest file at '/home/site/wwwroot/oryx-manifest.toml'
2020-11-27T17:36:09.583285560Z Could not find operation ID in manifest. Generating an operation id...
2020-11-27T17:36:09.583308261Z Build Operation ID: cf1c55c9-70a7-4bd1-bfa2-e3a2350cf36c
2020-11-27T17:36:10.816104146Z Writing output script to '/opt/startup/startup.sh'
2020-11-27T17:36:11.085709074Z Trying to find the startup DLL name...
2020-11-27T17:36:11.086560810Z Found the startup D name: Digibyte.API.dll
2020-11-27T17:36:11.088017070Z Running the command: dotnet "Digibyte.API.dll"
2020-11-27T17:36:47.812986572Z [41m[30mfail[39m[22m[49m: Microsoft.EntityFrameworkCore.Database.Command[20102]
2020-11-27T17:36:47.813053275Z Failed executing DbCommand (30,042ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
2020-11-27T17:36:47.813066676Z PRAGMA journal_mode = 'wal';
2020-11-27T17:36:47.944070916Z Unhandled exception. System.AggregateException: One or more errors occurred. (SQLite Error 5: 'database is locked'.)
**2020-11-27T17:36:47.944105917Z ---> Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 5: 'database is locked'.**
2020-11-27T17:36:47.944116518Z at Microsoft.Data.Sqlite.SqliteException.ThrowExceptionForRC(Int32 rc, sqlite3 db)
2020-11-27T17:36:47.944124918Z at Microsoft.Data.Sqlite.SqliteDataReader.NextResult()
2020-11-27T17:36:47.944132618Z at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader(CommandBehavior behavior)
2020-11-27T17:36:47.944140519Z at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader()
2020-11-27T17:36:47.944148119Z at Microsoft.Data.Sqlite.SqliteCommand.ExecuteNonQuery()
2020-11-27T17:36:47.944155719Z at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteNonQuery(RelationalCommandParameterObject parameterObject)
2020-11-27T17:36:47.944187521Z at Microsoft.EntityFrameworkCore.Sqlite.Storage.Internal.SqliteDatabaseCreator.Create()
2020-11-27T17:36:47.944197721Z at Microsoft.EntityFrameworkCore.Storage.RelationalDatabaseCreator.CreateAsync(CancellationToken cancellationToken)
2020-11-27T17:36:47.944205421Z at Microsoft.EntityFrameworkCore.Migrations.Internal.Migrator.MigrateAsync(String targetMigration, CancellationToken cancellationToken)
2020-11-27T17:36:47.944213022Z at Microsoft.Extensions.DependencyInjection.ServiceCollectionExtensions.CreateDatabase(IServiceProvider serviceProvider)
2020-11-27T17:36:47.944220422Z --- End of inner exception stack trace ---
2020-11-27T17:36:47.944229022Z at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
2020-11-27T17:36:47.944236223Z at System.Threading.Tasks.Task.Wait()
2020-11-27T17:36:47.944243123Z at Microsoft.Extensions.DependencyInjection.ServiceCollectionExtensions.AddHealthChecksUI(IServiceCollection services, String databaseName, Action`1 setupSettings)
2020-11-27T17:36:47.944250523Z at Digibyte.API.Startup.ConfigureServices(IServiceCollection services) in /home/vsts/work/1/s/Digibyte.API/Startup.cs:line 177
With the tip posted by #jason-pan I´ve found that the error was caused by HealthChecks package.
https://github.com/Xabaril/AspNetCore.Diagnostics.HealthChecks/issues/455
Related to: ASP.NET Core on Azure WebApp crashes because of SQLite
I am relatively new to gRPC service.
I am trying to deploy a Net core gRPC service into Linux docker container and accessing it from locally from VS console app.
I want to keep things as simple as possible thus the docker file is identical to a Net core gRPC docker file in VS where docker compose is pointing to it. When running the gRPC service in VS directly, the console app can access the service, just not in docker container.
gRPC launch settings
{
"profiles": {
"TrafficGrpc": {
"commandName": "Project",
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
},
"applicationUrl": "https://localhost:5001"
},
"Docker": {
"commandName": "Docker",
"launchUrl": "{Scheme}://{ServiceHost}:{ServicePort}",
"publishAllPorts": true,
"useSSL": true
}
}
}
gRPC app settings
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft": "Warning",
"Microsoft.Hosting.Lifetime": "Information"
}
},
"AllowedHosts": "*",
"Kestrel": {
"EndpointDefaults": {
"Protocols": "Http2"
}
}
}
Docker file
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1-buster-slim AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/core/sdk:3.1-buster AS build
WORKDIR /src
COPY ["gRPC/TrafficGrpc/TrafficGrpc.csproj", "TrafficGrpc/"]
RUN dotnet restore "TrafficGrpc/TrafficGrpc.csproj"
COPY . .
WORKDIR "/src/gRPC/TrafficGrpc"
RUN dotnet build "TrafficGrpc.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "TrafficGrpc.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "TrafficGrpc.dll"]
Docker compose file
version: "3.7"
services:
# Traffic service
traffic:
container_name: traffic
build:
context: .
dockerfile: Dockerfile-traffic
networks:
grpc_network:
environment:
- SERVICE_NAME=1
expose:
- "80"
- "443"
ports:
- "0.0.0.0:32773:80"
- "0.0.0.0:32774:443"
networks:
grpc_network:
Console app, VS
string trafficUrl = "http://localhost:32773";
//string trafficUrl = "https://localhost:32774";
Traffic traffic = new Traffic
{
Date = DateTime.Now.AddDays(1),
Area = Areas[rng.Next(Areas.Length)],
Condition = Conditions[rng.Next(Conditions.Length)]
};
GrpcChannel tChannel = GrpcChannel.ForAddress(trafficUrl);
TrafficCheckerClient tClient = new TrafficCheckerClient(tChannel);
TrafficConditionResponse tReply = await tClient.CheckTrafficConditionAsync(
new TrafficConditionRequest { Condition = traffic.Condition }); // <-- ERROR here
After running docker-compose file, the console app is just not able to connect to gRPC.
Using http, I get this error:
Grpc.Core.RpcException
HResult=0x80131500
Message=Status(StatusCode=Internal, Detail="Error starting gRPC call. HttpRequestException: An error occurred while sending the request. IOException: The response ended prematurely.")
Source=System.Private.CoreLib
StackTrace:
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
at ConsoleTestgRPC.Program.<Main>d__5.MoveNext() in D:\Workspace-GW-EV\CL\ConsoleTestgRPC\Program.cs:line 61
Using https, I get this error:
Grpc.Core.RpcException
HResult=0x80131500
Message=Status(StatusCode=Internal, Detail="Error starting gRPC call. HttpRequestException: The SSL connection could not be established, see inner exception. IOException: Authentication failed because the remote party has closed the transport stream.")
Source=System.Private.CoreLib
StackTrace:
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
at ConsoleTestgRPC.Program.<Main>d__5.MoveNext() in D:\Workspace-GW-EV\CL\ConsoleTestgRPC\Program.cs:line 61
At this point, I do not know whether it is a docker network or a gRPC config issue.
I need help to point me in the right direction.
THANK YOU
I meet familiar problem today.
You set EXPOSE 80 in the dockerfile, so you need to make grpc to listen the port which bind to it.
So in my case, the grpc listen to the 8099, and expose it to host port: 32812, then client creat channel to it. And it work.
In dockerfile:
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1-buster-slim AS base
WORKDIR /app
EXPOSE 8099
EXPOSE 443
In server to listen port:
public static IWebHostBuilder CreateHostBuilder(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.ConfigureKestrel(options =>
{
options.ListenAnyIP(
8099,
listenOptions => { listenOptions.Protocols = HttpProtocols.Http2; }
);
})
.UseStartup<Startup>();
In client to create channel:
GrpcClientFactory.AllowUnencryptedHttp2 = true;
using var http = GrpcChannel.ForAddress("http://localhost:32812");
Binding port
May this will help you, good luck~
I Had some problems to run the image on kubernetes. To Fix:
1 - To expose port i had to change on image and appliation and expose the Enviroment var(but only EXPOSE works too).
Force the port
2 - My issue was the certificate on TLS protocol (i did't had). I has publicated on my github how i fixed it without certificate. But take atention on the enviroment. This exemple can be applied only on internal comunication with ClusterIp. It's a insecure implementation mode to public exposition. 😉🤘
link fix(example)
https://github.com/davidsonsilvadev/grpc-core3.1-docker-kubernetes-without-certificate
I have a service (an ASP.NET Core Web application) that depends on MSSQL. The services are orchestrated using Docker compose, and I want docker compose to first start the database and wait for it to be ready before running my service. For that, I am defining the docker-compose.yml as:
version: '3.7'
services:
sql.data:
container_name: db_service
image: microsoft/mssql-server-linux:2017-latest
healthcheck:
test: ["CMD", "/opt/mssql-tools/bin/sqlcmd", "-S", "http://localhost:1433", "-U", "sa", "-P", "Pass_word", "-Q", "SELECT 1", "||", "exit 1"]
my_service:
container_name: my_service_container
image: ${DOCKER_REGISTRY-}my_service
build:
context: .
dockerfile: MyService/Dockerfile
depends_on:
- sql.data
With this health-check, Docker compose does not wait for the database service to be ready, and starts my_service immediately after, and, as expected, my_service fails connecting to the database. Part of the log is:
Recreating db_service ... done
Recreating my_service_container ... done
Attaching to db_service, my_service_container
my_service_container | info: ...Context[0]
my_service_container | Migrating database associated with context Context
my_service_container | info: Microsoft.EntityFrameworkCore.Infrastructure[10403]
my_service_container | Entity Framework Core 3.1.1 initialized 'Context' using provider 'Microsoft.EntityFrameworkCore.SqlServer' with options: MigrationsAssembly=MyService
my_service_container | fail: Context[0]
my_service_container | An error occurred while migrating the database used on context Context
my_service_container | Microsoft.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: TCP Provider, error: 40 - Could not open a connection to SQL Server)
...
exception details
...
my_service_container | ClientConnectionId:00000000-0000-0000-0000-000000000000
my_service_container exited with code 0
db_service | 2020-03-05 05:45:51.82 Server Microsoft SQL Server 2017 (RTM-CU13) (KB4466404) - 14.0.3048.4 (X64)
Nov 30 2018 12:57:58
Copyright (C) 2017 Microsoft Corporation
Developer Edition (64-bit) on Linux (Ubuntu 16.04.5 LTS)
2020-03-05 05:45:51.82 Server UTC adjustment: 0:00
2020-03-05 05:45:51.82 Server (c) Microsoft Corporation.
2020-03-05 05:45:51.82 Server All rights reserved.
2020-03-05 05:45:51.82 Server Server process ID is 4120.
2020-03-05 05:45:51.82 Server Logging SQL Server messages in file '/var/opt/mssql/log/errorlog'.
2020-03-05 05:45:51.82 Server Registry startup parameters:
-d /var/opt/mssql/data/master.mdf
-l /var/opt/mssql/data/mastlog.ldf
-e /var/opt/mssql/log/errorlog
As shown in the logs, the docker compose first starts the DB, but does not wait for it become ready before running my service.
I tried different syntax for the healthcheck, e.g.,:
test: /opt/mssql-tools/bin/sqlcmd -S http://localhost:1433 -U sa -P ${SA_PASSWORD} -Q "SELECT 1" || exit 1
But neither worked as expected.
I have checked the following sources online, but using neither I was able to solve the problem:
stackoverflow answer
github comment
github sample
Is this functionality even supported in version 3.7? because of this confusing comment
Question
Any thoughts on how best I can wait for MSSQL service to start?
After searching and trying many different scenarios, I was able to add waiting using the following composer file. This is for asp.net core solution. The key is that you have to overwrite entrypoint if it is specified in dockerfile. Also, you need to make sure to save "wait-for-it.sh" LF as line ending instead of CRLF, otherwise you'll get the error of file not found.
The dockerfile should have the following (download it from here: https://raw.githubusercontent.com/vishnubob/wait-for-it/master/wait-for-it.sh, make sure to save the file):
COPY ./wait-for-it.sh /wait-for-it.sh
RUN chmod +x wait-for-it.sh
docker-compose.yml
version: '3.7'
services:
vc-db:
image: mcr.microsoft.com/mssql/server:latest
ports:
- "${DOCKER_SQL_PORT:-1433}:1433"
expose:
- 1433
environment:
- ACCEPT_EULA=Y
- MSSQL_PID=Express
- SA_PASSWORD=v!rto_Labs!
networks:
- virto
vc-platform-web:
image: virtocommerce/platform:${DOCKER_TAG:-latest}
ports:
- "${DOCKER_PLATFORM_PORT:-8090}:80"
environment:
- ASPNETCORE_URLS=http://+
depends_on:
- vc-db
entrypoint: ["/wait-for-it.sh", "vc-db:1433", "-t", "120", "--", "dotnet", "VirtoCommerce.Platform.Web.dll"]
networks:
- virto
When you use depends_on, docker-compose will just launch your base service with more priority and never wait for start services.
There are some useful external program that help you to wait for specific service (port), then run another service.
vishnubob/wait-for-it is one of them which blocks execution flow until your specific port(s) get ready.
Another good choice is eficode/wait-for which already prepared for docker-compose.
Example usage (according to eficode/wait-for docs)
version: '2'
services:
db:
image: postgres:9.4
backend:
build: backend
# Blocks execution flow util db:5432 is ready (Or you can use localhost instead)
command: sh -c './wait-for db:5432 -- npm start'
depends_on:
- db
-- UPDATE --
Consider you have a Python application that depend on a database like PostgreSQL, and also your application will run with this command: python app.py
As Official Docker Document said, Put vishnubob/wait-for-it in your image (inside of your other project files like app.py)
Now just put this lines in your docker-compose.yml:
version: "3"
services:
web:
build: .
ports:
- "80:8000"
depends_on:
- "db"
# This command waits until `db:5432` respond (5432 is default PostgreSQL port)
# then runs our application by this command: `python app.py`
command: ["./wait-for-it.sh", "db:5432", "--", "python", "app.py"]
db:
image: postgres
Note: Don't forget to put this commands in your Dockerfile inside your image files:
# Copy wait-for-it.sh into our image
COPY wait-for-it.sh wait-for-it.sh
# Make it executable, in Linux
RUN chmod +x wait-for-it.sh
Create two separate dockerfiles (e.g):
Mssql.Dockerfile
App.Dockerfile
Set up the sequence within docker-compose.yml
Mssql.Dockerfile
FROM mcr.microsoft.com/mssql/server AS base
ENV ACCEPT_EULA=Y
ENV SA_PASSWORD=Password123
COPY . .
COPY ["Db/Scripts/*", "Db/Scripts/"]
VOLUME ./Db:/var/opt/mssql/data
HEALTHCHECK --interval=10s --timeout=5s --start-period=10s --retries=10 \
CMD /opt/mssql-tools/bin/sqlcmd -S . -U sa -P Password123 -i Db/Scripts/SetupDb.sql || exit 1
App.Dockerfile:
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1-buster-slim AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/core/sdk:3.1-buster AS build
WORKDIR /src
COPY ["AspNetCoreWebApplication/AspNetCoreWebApplication.csproj", "AspNetCoreWebApplication/"]
COPY ["WebApp.Data.EF/WebApp.Data.EF.csproj", "WebApp.Data.EF/"]
COPY ["WebApp.Service/WebApp.Service.csproj", "WebApp.Service/"]
RUN dotnet restore "AspNetCoreWebApplication/AspNetCoreWebApplication.csproj"
COPY . .
WORKDIR "/src/AspNetCoreWebApplication"
RUN dotnet build "AspNetCoreWebApplication.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "AspNetCoreWebApplication.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "AspNetCoreWebApplication.dll"]
Docker-compose.yml:
version: '3.7'
services:
api:
image: aspnetcore/mentoring_api
container_name: mentoring_api
build:
context: .
dockerfile: App.Dockerfile
ports:
- 8081:80
expose:
- 8081
environment:
ASPNETCORE_ENVIRONMENT: Development
depends_on:
- sqlserver
sqlserver:
image: aspnetcore/mentoring_db
container_name: mentoring_db
build:
context: .
dockerfile: Mssql.Dockerfile
ports:
- "1433:1433"
expose:
- 1433
environment:
- ACCEPT_EULA=Y
- SA_PASSWORD=Password123
volumes:
- ./Db:/var/opt/mssql/data
Note:
The connection string will look like: "Server=sqlserver;Database=Northwind;Trusted_Connection=False;User Id=sa;Password=Password123;MultipleActiveResultSets=true"
Here is a complete example
version: "3.8"
services:
ms-db-server:
image: mcr.microsoft.com/mssql/server
environment:
- SA_PASSWORD=P#ssw0rd
- ACCEPT_EULA=Y
volumes:
- ./data/db/mssql/scripts:/scripts/
ports:
- "1433:1433"
#entrypoint: /bin/bash
command:
- /bin/bash
- -c
- |
/opt/mssql/bin/sqlservr &
pid=$$!
echo "Waiting for MS SQL to be available ⏳"
/opt/mssql-tools/bin/sqlcmd -l 30 -S localhost -h-1 -V1 -U sa -P $$SA_PASSWORD -Q "SET NOCOUNT ON SELECT \"YAY WE ARE UP\" , ##servername"
is_up=$$?
while [ $$is_up -ne 0 ] ; do
echo -e $$(date)
/opt/mssql-tools/bin/sqlcmd -l 30 -S localhost -h-1 -V1 -U sa -P $$SA_PASSWORD -Q "SET NOCOUNT ON SELECT \"YAY WE ARE UP\" , ##servername"
is_up=$$?
sleep 5
done
for foo in /scripts/*.sql
do /opt/mssql-tools/bin/sqlcmd -U sa -P $$SA_PASSWORD -l 30 -e -i $$foo
done
echo "All scripts have been executed. Waiting for MS SQL(pid $$pid) to terminate."
wait $$pid
tempo:
image: grafana/tempo:latest
command: ["-config.file=/etc/tempo.yaml"]
volumes:
- ./etc/tempo-local.yaml:/etc/tempo.yaml
- ./data/tempo-data:/tmp/tempo
ports:
- "14268" # jaeger ingest, Jaeger - Thrift HTTP
- "14250" # Jaeger - GRPC
- "55680" # OpenTelemetry
- "3100" # tempo
- "6831/udp" # Jaeger - Thrift Compact
- "6832/udp" # Jaeger - Thrift Binary
tempo-query:
image: grafana/tempo-query:latest
command: ["--grpc-storage-plugin.configuration-file=/etc/tempo-query.yaml"]
volumes:
- ./etc/tempo-query.yaml:/etc/tempo-query.yaml
ports:
- "16686:16686" # jaeger-ui
depends_on:
- tempo
loki:
image: grafana/loki:2.1.0
command: -config.file=/etc/loki/loki-local.yaml
ports:
- "3101:3100" # loki needs to be exposed so it receives logs
environment:
- JAEGER_AGENT_HOST=tempo
- JAEGER_ENDPOINT=http://tempo:14268/api/traces # send traces to Tempo
- JAEGER_SAMPLER_TYPE=const
- JAEGER_SAMPLER_PARAM=1
volumes:
- ./etc/loki-local.yaml:/etc/loki/loki-local.yaml
- ./data/loki-data:/tmp/loki
nodejs-otel-tempo-api:
build: .
command: './wait-for.sh ms-db-server:1433 -- node ./dist/server.js'
ports:
- "5555:5555"
environment:
- OTEL_EXPORTER_JAEGER_ENDPOINT=http://tempo:14268/api/traces
- OTEL_SERVICE_NAME=nodejs-opentelemetry-tempo
- LOG_FILE_NAME=/app/logs/nodejs-opentelemetry-tempo.log
- DB_USER=sa
- DB_PASS=P#ssw0rd
- DB_SERVER=ms-db-server
- DB_NAME=OtelTempo
volumes:
- ./data/logs:/app/logs
- ./etc/wait-for.sh:/app/bin/wait-for.sh #https://github.com/eficode/wait-for
depends_on:
- ms-db-server
- tempo-query
promtail:
image: grafana/promtail:master-ee9c629
command: -config.file=/etc/promtail/promtail-local.yaml
volumes:
- ./etc/promtail-local.yaml:/etc/promtail/promtail-local.yaml
- ./data/logs:/app/logs
depends_on:
- nodejs-otel-tempo-api
- loki
prometheus:
image: prom/prometheus:latest
volumes:
- ./etc/prometheus.yaml:/etc/prometheus.yaml
entrypoint:
- /bin/prometheus
- --config.file=/etc/prometheus.yaml
ports:
- "9090:9090"
depends_on:
- nodejs-otel-tempo-api
grafana:
image: grafana/grafana:7.4.0-ubuntu
volumes:
- ./data/grafana-data/datasources:/etc/grafana/provisioning/datasources
- ./data/grafana-data/dashboards-provisioning:/etc/grafana/provisioning/dashboards
- ./data/grafana-data/dashboards:/var/lib/grafana/dashboards
environment:
- GF_AUTH_ANONYMOUS_ENABLED=true
- GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
- GF_AUTH_DISABLE_LOGIN_FORM=true
ports:
- "3000:3000"
depends_on:
- prometheus
- tempo-query
- loki
You can write a simple script, which will be launched in container with your app. For example, you can just set a delay, using sleep N ( where N it is a time which is needed for starting your DB), or you can use a until cycle in which you to can try to connect to your DB, and when it will be possible, then you can start your app.
I know this is not a perfect solution, but it helped me when I had a similar problem
Using Kubernetes Deployment, the following probe successfullly identified the ready state:
containers:
- name: mssql
image: mcr.microsoft.com/mssql/server:2019-latest
ports:
- containerPort: 1433
env:
- name: ACCEPT_EULA
value: 'Y'
startupProbe:
exec:
command:
- /bin/sh
- '-c'
- '/opt/mssql-tools/bin/sqlcmd -U sa -P "${SA_PASSWORD}" -Q "SELECT \"READY\"" | grep -q "READY"'
failureThreshold: 15
periodSeconds: 10
You could code this as a livenessProbe with initialDelay if your Kubernetes version is <1.20.
Explanation:
sqlcmd returns a '0' status irrespective of whether the query returns anything or not. However, grep -q returns 0 or 1 depending on presence of the word "READY".
I have not used docker-compose but I suspect this command would work as a healthcheck test (assuming SA_PASSWORD is injected into the environment), ie:
healthcheck:
test:
- /bin/sh
- '-c'
- '/opt/mssql-tools/bin/sqlcmd -U sa -P "${SA_PASSWORD}" -Q "SELECT \"READY\"" | grep -q "READY"'
Configure Liveness, Readiness and Startup Probes
Pod Lifecycle
you can also delay the docker startup until mssql is up:
docker-compose.yaml
mssql:
image: mcr.microsoft.com/mssql/server:2017-latest
ports:
- 1433:1433
environment:
SA_PASSWORD: "t9D4:EHfU6Xgccs-"
ACCEPT_EULA: "Y"
networks:
- backend
command:
- /bin/bash
- -c
- |
/opt/mssql/bin/sqlservr
curl -s https://raw.githubusercontent.com/vishnubob/wait-for-it/master/wait-for-it.sh | bash /dev/stdin localhost:1433