Hi everyone, I have a problem with Temporal and .NET on Linux(Docker).
I use Temporal for my messaging operation(SMS, Email, and Slack). On my computer(OS win 11, .NET core 6) everything is ok. But when I deploy to the server(Azure kuber, using docker) I face with the following error. Can anyone help me?
[2023-03-31T14:31:50.6649230+00:00 ERR] [(5497) e18956a2-3af2-4398-af87-50d3692a2da6 155.138.155.80 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.54] (NotificationHelper.Service.NotificationSendService) Error in sending Slack message using Temporal
System.DllNotFoundException: Unable to load shared library 'temporal_sdk_bridge' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libtemporal_sdk_bridge: cannot open shared object file: No such file or directory
at Temporalio.Bridge.Interop.Methods.runtime_new(RuntimeOptions* options)
at Temporalio.Bridge.Runtime..ctor(TemporalRuntimeOptions options)
at Temporalio.Runtime.TemporalRuntime..ctor(TemporalRuntimeOptions options)
at Temporalio.Runtime.TemporalRuntime.<>c.<.cctor>b__9_0()
at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
--- End of stack trace from previous location ---
at System.Lazy`1.CreateValue()
at Temporalio.Runtime.TemporalRuntime.get_Default()
at Temporalio.Client.TemporalConnection.ConnectAsync(TemporalConnectionOptions options)
at Temporalio.Client.TemporalClient.ConnectAsync(TemporalClientConnectOptions options)
at NotificationHelper.Service.NotificationSendService.SendSlackNotification(SlackNotificationInputModel notificationInputModel) in /src/common/NotificationHelper/Service/NotificationSendService.cs:line 212
I have developed the workflow using Python and everything is ok with that.
To confirm, it runs in one environment but not another? I suspect the temporal_sdk_bridge.dll is not getting deployed alongside the Temporalio.dll. Can you check the environment differences with regards to the presence of this bridge DLL? We embed it in the NuGet package per environment.
(note it may be libtemporal_sdk_bridge.so or similar in Linux)
Yes, exactly. It runs on my local environment(my laptop) and works properly.
I checked the prod env(Linux docker), and I can find the Temporalio.dll but not temporal_sdk_bridge.dll.
I searched the dir using regex and only one match find: Temporalio.dll
I am unaware of how you are deploying or adding the Temporalio dependency in docker, but you need to make sure libtemporal_sdk_bridge.so comes along with it. NuGet does this for you by default which is why it works locally. Notice when you do a dotnet build, in bin/<Debug|Release>/<net6.0|other> there is a runtimes folder. This contains the bridge shared libraries that are needed per os and arch (and if you have any other dependencies that use external shared libraries, they are in here too). You can bring over the entire runtimes folder, or just runtimes/linux-x64/native/libtemporal_sdk_bridge.so if just wanted Linux x64.
I checked and this file is there. I enabled the LD_DEBUG env and I got these logs.
System.DllNotFoundException: Unable to load shared library 'temporal_sdk_bridge' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libtemporal_sdk_bridge: cannot open shared object file: No such file or directory
at Temporalio.Bridge.Interop.Methods.runtime_new(RuntimeOptions* options)
at Temporalio.Bridge.Runtime..ctor(TemporalRuntimeOptions options)
at Temporalio.Runtime.TemporalRuntime..ctor(TemporalRuntimeOptions options)
at Temporalio.Runtime.TemporalRuntime.<>c.<.cctor>b__9_0()
at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
--- End of stack trace from previous location ---
at System.Lazy`1.CreateValue()
at System.Lazy`1.get_Value()
at Temporalio.Runtime.TemporalRuntime.get_Default()
at Temporalio.Client.TemporalConnection.ConnectAsync(TemporalConnectionOptions options)
at Temporalio.Client.TemporalClient.ConnectAsync(TemporalClientConnectOptions options)
at NotificationHelper.Service.NotificationSendService.SendSMS(String toPhoneNumber, String messageBody) in /src/common/NotificationHelper/Service/NotificationSendService.cs:line 75
[2023-03-31T17:01:05.5641139+00:00 ERR] [(5505) 159b4689-a3f8-45b7-8588-06bcec17fb6f 155.138.155.80 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.54] (NotificationHelper.Service.NotificationSendService) Temporal send Email
System.DllNotFoundException: Unable to load shared library 'temporal_sdk_bridge' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libtemporal_sdk_bridge: cannot open shared object file: No such file or directory
at Temporalio.Bridge.Interop.Methods.runtime_new(RuntimeOptions* options)
at Temporalio.Bridge.Runtime..ctor(TemporalRuntimeOptions options)
at Temporalio.Runtime.TemporalRuntime..ctor(TemporalRuntimeOptions options)
at Temporalio.Runtime.TemporalRuntime.<>c.<.cctor>b__9_0()
at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
--- End of stack trace from previous location ---
at System.Lazy`1.CreateValue()
at System.Lazy`1.get_Value()
at Temporalio.Runtime.TemporalRuntime.get_Default()
at Temporalio.Client.TemporalConnection.ConnectAsync(TemporalConnectionOptions options)
at Temporalio.Client.TemporalClient.ConnectAsync(TemporalClientConnectOptions options)
at NotificationHelper.Service.NotificationSendService.SendGeneralEmail(String emailAddress, String subject, String body) in /src/common/NotificationHelper/Service/NotificationSendService.cs:line 122
[2023-03-31T17:01:05.5664620+00:00 ERR] [(5505) 159b4689-a3f8-45b7-8588-06bcec17fb6f 155.138.155.80 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.54] (NotificationHelper.Service.NotificationSendService) Temporal send SMS
System.DllNotFoundException: Unable to load shared library 'temporal_sdk_bridge' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libtemporal_sdk_bridge: cannot open shared object file: No such file or directory
at Temporalio.Bridge.Interop.Methods.runtime_new(RuntimeOptions* options)
at Temporalio.Bridge.Runtime..ctor(TemporalRuntimeOptions options)
at Temporalio.Runtime.TemporalRuntime..ctor(TemporalRuntimeOptions options)
at Temporalio.Runtime.TemporalRuntime.<>c.<.cctor>b__9_0()
at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
--- End of stack trace from previous location ---
at System.Lazy`1.CreateValue()
at System.Lazy`1.get_Value()
at Temporalio.Runtime.TemporalRuntime.get_Default()
at Temporalio.Client.TemporalConnection.ConnectAsync(TemporalConnectionOptions options)
at Temporalio.Client.TemporalClient.ConnectAsync(TemporalClientConnectOptions options)
at NotificationHelper.Service.NotificationSendService.SendSMS(String toPhoneNumber, String messageBody) in /src/common/NotificationHelper/Service/NotificationSendService.cs:line 75
1:
1: file=Microsoft.DiaSymReader.Native.amd64.dll [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1: find library=Microsoft.DiaSymReader.Native.amd64.dll [0]; searching
1: search cache=/etc/ld.so.cache
1: search path=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/lib:/usr/lib (system search path)
1: trying file=/lib/x86_64-linux-gnu/Microsoft.DiaSymReader.Native.amd64.dll
1: trying file=/usr/lib/x86_64-linux-gnu/Microsoft.DiaSymReader.Native.amd64.dll
1: trying file=/lib/Microsoft.DiaSymReader.Native.amd64.dll
1: trying file=/usr/lib/Microsoft.DiaSymReader.Native.amd64.dll
1:
1:
1: file=/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/Microsoft.DiaSymReader.Native.amd64.dll [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=Microsoft.DiaSymReader.Native.amd64.dll [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1: find library=Microsoft.DiaSymReader.Native.amd64.dll [0]; searching
1: search cache=/etc/ld.so.cache
1: search path=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/lib:/usr/lib (system search path)
1: trying file=/lib/x86_64-linux-gnu/Microsoft.DiaSymReader.Native.amd64.dll
1: trying file=/usr/lib/x86_64-linux-gnu/Microsoft.DiaSymReader.Native.amd64.dll
1: trying file=/lib/Microsoft.DiaSymReader.Native.amd64.dll
1: trying file=/usr/lib/Microsoft.DiaSymReader.Native.amd64.dll
1:
1:
1: file=/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/Microsoft.DiaSymReader.Native.amd64.dll [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
[2023-03-31T17:01:05.5792392+00:00 ERR] [(5505) 159b4689-a3f8-45b7-8588-06bcec17fb6f 155.138.155.80 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.54] (Microsoft.EntityFrameworkCore.Query) An exception occurred while iterating over the results of a query for context type '"Doctors.Repository.DoctorContext"'."\n""System.InvalidOperationException: A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913.\n at Microsoft.EntityFrameworkCore.Infrastructure.Internal.ConcurrencyDetector.EnterCriticalSection()\n at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()"
System.InvalidOperationException: A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913.
at Microsoft.EntityFrameworkCore.Infrastructure.Internal.ConcurrencyDetector.EnterCriticalSection()
at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
1:
1: file=/app/runtimes/linux-x64/native/temporal_sdk_bridge.so [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/temporal_sdk_bridge.so [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=/app/temporal_sdk_bridge.so [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=temporal_sdk_bridge.so [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1: find library=temporal_sdk_bridge.so [0]; searching
1: search cache=/etc/ld.so.cache
1: search path=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/lib:/usr/lib (system search path)
1: trying file=/lib/x86_64-linux-gnu/temporal_sdk_bridge.so
1: trying file=/usr/lib/x86_64-linux-gnu/temporal_sdk_bridge.so
1: trying file=/lib/temporal_sdk_bridge.so
1: trying file=/usr/lib/temporal_sdk_bridge.so
1:
1:
1: file=/app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1: file=/app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so [0]; generating link map
1: dynamic: 0x00007fd92b4afd40 base: 0x00007fd92a07f000 size: 0x000000000143d768
1: entry: 0x00007fd92a07f000 phdr: 0x00007fd92a07f040 phnum: 10
1:
1: checking for version `GLIBC_2.3' in file /lib64/ld-linux-x86-64.so.2 [0] required by file /app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so [0]
1: checking for version `GLIBC_2.2.5' in file /lib/x86_64-linux-gnu/libm.so.6 [0] required by file /app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so [0]
1: checking for version `GCC_3.3' in file /lib/x86_64-linux-gnu/libgcc_s.so.1 [0] required by file /app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so [0]
1: checking for version `GCC_4.2.0' in file /lib/x86_64-linux-gnu/libgcc_s.so.1 [0] required by file /app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so [0]
1: checking for version `GCC_3.0' in file /lib/x86_64-linux-gnu/libgcc_s.so.1 [0] required by file /app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so [0]
1: checking for version `GLIBC_2.7' in file /lib/x86_64-linux-gnu/libc.so.6 [0] required by file /app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so [0]
1: checking for version `GLIBC_2.32' in file /lib/x86_64-linux-gnu/libc.so.6 [0] required by file /app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so [0]
1: /lib/x86_64-linux-gnu/libc.so.6: error: version lookup error: version `GLIBC_2.32' not found (required by /app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so) (fatal)
1:
1: file=/app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so [0]; destroying link map
1:
1: file=/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libtemporal_sdk_bridge.so [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=/app/libtemporal_sdk_bridge.so [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=libtemporal_sdk_bridge.so [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1: find library=libtemporal_sdk_bridge.so [0]; searching
1: search cache=/etc/ld.so.cache
1: search path=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/lib:/usr/lib (system search path)
1: trying file=/lib/x86_64-linux-gnu/libtemporal_sdk_bridge.so
1: trying file=/usr/lib/x86_64-linux-gnu/libtemporal_sdk_bridge.so
1: trying file=/lib/libtemporal_sdk_bridge.so
1: trying file=/usr/lib/libtemporal_sdk_bridge.so
1:
1:
1: file=/app/runtimes/linux-x64/native/temporal_sdk_bridge [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/temporal_sdk_bridge [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=/app/temporal_sdk_bridge [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=temporal_sdk_bridge [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1: find library=temporal_sdk_bridge [0]; searching
1: search cache=/etc/ld.so.cache
1: search path=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/lib:/usr/lib (system search path)
1: trying file=/lib/x86_64-linux-gnu/temporal_sdk_bridge
1: trying file=/usr/lib/x86_64-linux-gnu/temporal_sdk_bridge
1: trying file=/lib/temporal_sdk_bridge
1: trying file=/usr/lib/temporal_sdk_bridge
1:
1:
1: file=/app/runtimes/linux-x64/native/libtemporal_sdk_bridge [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libtemporal_sdk_bridge [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=/app/libtemporal_sdk_bridge [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=libtemporal_sdk_bridge [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1: find library=libtemporal_sdk_bridge [0]; searching
1: search cache=/etc/ld.so.cache
1: search path=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/lib:/usr/lib (system search path)
1: trying file=/lib/x86_64-linux-gnu/libtemporal_sdk_bridge
1: trying file=/usr/lib/x86_64-linux-gnu/libtemporal_sdk_bridge
1: trying file=/lib/libtemporal_sdk_bridge
1: trying file=/usr/lib/libtemporal_sdk_bridge
1:
Unhandled exception. System.DllNotFoundException: Unable to load shared library 'temporal_sdk_bridge' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libtemporal_sdk_bridge: cannot open shared object file: No such file or directory
at Temporalio.Bridge.Interop.Methods.runtime_free(Runtime* runtime)
at Temporalio.Bridge.Runtime.ReleaseHandle()
at System.Runtime.InteropServices.SafeHandle.InternalRelease(Boolean disposeOrFinalizeOperation)
at System.Runtime.InteropServices.SafeHandle.Dispose(Boolean disposing)
at System.Runtime.InteropServices.SafeHandle.Finalize()
Any idea @Chad_Retz ?
I checked and this file is there. I enabled the LD_DEBUG env and I got these logs.
System.DllNotFoundException: Unable to load shared library 'temporal_sdk_bridge' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libtemporal_sdk_bridge: cannot open shared object file: No such file or directory
at Temporalio.Bridge.Interop.Methods.runtime_new(RuntimeOptions* options)
at Temporalio.Bridge.Runtime..ctor(TemporalRuntimeOptions options)
at Temporalio.Runtime.TemporalRuntime..ctor(TemporalRuntimeOptions options)
at Temporalio.Runtime.TemporalRuntime.<>c.<.cctor>b__9_0()
at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
--- End of stack trace from previous location ---
at System.Lazy`1.CreateValue()
at System.Lazy`1.get_Value()
at Temporalio.Runtime.TemporalRuntime.get_Default()
at Temporalio.Client.TemporalConnection.ConnectAsync(TemporalConnectionOptions options)
at Temporalio.Client.TemporalClient.ConnectAsync(TemporalClientConnectOptions options)
at NotificationHelper.Service.NotificationSendService.SendSMS(String toPhoneNumber, String messageBody) in /src/common/NotificationHelper/Service/NotificationSendService.cs:line 75
[2023-03-31T17:01:05.5641139+00:00 ERR] [(5505) 159b4689-a3f8-45b7-8588-06bcec17fb6f 155.138.155.80 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.54] (NotificationHelper.Service.NotificationSendService) Temporal send Email
System.DllNotFoundException: Unable to load shared library 'temporal_sdk_bridge' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libtemporal_sdk_bridge: cannot open shared object file: No such file or directory
at Temporalio.Bridge.Interop.Methods.runtime_new(RuntimeOptions* options)
at Temporalio.Bridge.Runtime..ctor(TemporalRuntimeOptions options)
at Temporalio.Runtime.TemporalRuntime..ctor(TemporalRuntimeOptions options)
at Temporalio.Runtime.TemporalRuntime.<>c.<.cctor>b__9_0()
at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
--- End of stack trace from previous location ---
at System.Lazy`1.CreateValue()
at System.Lazy`1.get_Value()
at Temporalio.Runtime.TemporalRuntime.get_Default()
at Temporalio.Client.TemporalConnection.ConnectAsync(TemporalConnectionOptions options)
at Temporalio.Client.TemporalClient.ConnectAsync(TemporalClientConnectOptions options)
at NotificationHelper.Service.NotificationSendService.SendGeneralEmail(String emailAddress, String subject, String body) in /src/common/NotificationHelper/Service/NotificationSendService.cs:line 122
[2023-03-31T17:01:05.5664620+00:00 ERR] [(5505) 159b4689-a3f8-45b7-8588-06bcec17fb6f 155.138.155.80 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.54] (NotificationHelper.Service.NotificationSendService) Temporal send SMS
System.DllNotFoundException: Unable to load shared library 'temporal_sdk_bridge' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libtemporal_sdk_bridge: cannot open shared object file: No such file or directory
at Temporalio.Bridge.Interop.Methods.runtime_new(RuntimeOptions* options)
at Temporalio.Bridge.Runtime..ctor(TemporalRuntimeOptions options)
at Temporalio.Runtime.TemporalRuntime..ctor(TemporalRuntimeOptions options)
at Temporalio.Runtime.TemporalRuntime.<>c.<.cctor>b__9_0()
at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
--- End of stack trace from previous location ---
at System.Lazy`1.CreateValue()
at System.Lazy`1.get_Value()
at Temporalio.Runtime.TemporalRuntime.get_Default()
at Temporalio.Client.TemporalConnection.ConnectAsync(TemporalConnectionOptions options)
at Temporalio.Client.TemporalClient.ConnectAsync(TemporalClientConnectOptions options)
at NotificationHelper.Service.NotificationSendService.SendSMS(String toPhoneNumber, String messageBody) in /src/common/NotificationHelper/Service/NotificationSendService.cs:line 75
1:
1: file=Microsoft.DiaSymReader.Native.amd64.dll [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1: find library=Microsoft.DiaSymReader.Native.amd64.dll [0]; searching
1: search cache=/etc/ld.so.cache
1: search path=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/lib:/usr/lib (system search path)
1: trying file=/lib/x86_64-linux-gnu/Microsoft.DiaSymReader.Native.amd64.dll
1: trying file=/usr/lib/x86_64-linux-gnu/Microsoft.DiaSymReader.Native.amd64.dll
1: trying file=/lib/Microsoft.DiaSymReader.Native.amd64.dll
1: trying file=/usr/lib/Microsoft.DiaSymReader.Native.amd64.dll
1:
1:
1: file=/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/Microsoft.DiaSymReader.Native.amd64.dll [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=Microsoft.DiaSymReader.Native.amd64.dll [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1: find library=Microsoft.DiaSymReader.Native.amd64.dll [0]; searching
1: search cache=/etc/ld.so.cache
1: search path=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/lib:/usr/lib (system search path)
1: trying file=/lib/x86_64-linux-gnu/Microsoft.DiaSymReader.Native.amd64.dll
1: trying file=/usr/lib/x86_64-linux-gnu/Microsoft.DiaSymReader.Native.amd64.dll
1: trying file=/lib/Microsoft.DiaSymReader.Native.amd64.dll
1: trying file=/usr/lib/Microsoft.DiaSymReader.Native.amd64.dll
1:
1:
1: file=/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/Microsoft.DiaSymReader.Native.amd64.dll [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
[2023-03-31T17:01:05.5792392+00:00 ERR] [(5505) 159b4689-a3f8-45b7-8588-06bcec17fb6f 155.138.155.80 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.54] (Microsoft.EntityFrameworkCore.Query) An exception occurred while iterating over the results of a query for context type '"Doctors.Repository.DoctorContext"'."\n""System.InvalidOperationException: A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913.\n at Microsoft.EntityFrameworkCore.Infrastructure.Internal.ConcurrencyDetector.EnterCriticalSection()\n at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()"
System.InvalidOperationException: A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913.
at Microsoft.EntityFrameworkCore.Infrastructure.Internal.ConcurrencyDetector.EnterCriticalSection()
at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
1:
1: file=/app/runtimes/linux-x64/native/temporal_sdk_bridge.so [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/temporal_sdk_bridge.so [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=/app/temporal_sdk_bridge.so [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=temporal_sdk_bridge.so [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1: find library=temporal_sdk_bridge.so [0]; searching
1: search cache=/etc/ld.so.cache
1: search path=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/lib:/usr/lib (system search path)
1: trying file=/lib/x86_64-linux-gnu/temporal_sdk_bridge.so
1: trying file=/usr/lib/x86_64-linux-gnu/temporal_sdk_bridge.so
1: trying file=/lib/temporal_sdk_bridge.so
1: trying file=/usr/lib/temporal_sdk_bridge.so
1:
1:
1: file=/app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1: file=/app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so [0]; generating link map
1: dynamic: 0x00007fd92b4afd40 base: 0x00007fd92a07f000 size: 0x000000000143d768
1: entry: 0x00007fd92a07f000 phdr: 0x00007fd92a07f040 phnum: 10
1:
1: checking for version `GLIBC_2.3' in file /lib64/ld-linux-x86-64.so.2 [0] required by file /app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so [0]
1: checking for version `GLIBC_2.2.5' in file /lib/x86_64-linux-gnu/libm.so.6 [0] required by file /app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so [0]
1: checking for version `GCC_3.3' in file /lib/x86_64-linux-gnu/libgcc_s.so.1 [0] required by file /app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so [0]
1: checking for version `GCC_4.2.0' in file /lib/x86_64-linux-gnu/libgcc_s.so.1 [0] required by file /app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so [0]
1: checking for version `GCC_3.0' in file /lib/x86_64-linux-gnu/libgcc_s.so.1 [0] required by file /app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so [0]
1: checking for version `GLIBC_2.7' in file /lib/x86_64-linux-gnu/libc.so.6 [0] required by file /app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so [0]
1: checking for version `GLIBC_2.32' in file /lib/x86_64-linux-gnu/libc.so.6 [0] required by file /app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so [0]
1: /lib/x86_64-linux-gnu/libc.so.6: error: version lookup error: version `GLIBC_2.32' not found (required by /app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so) (fatal)
1:
1: file=/app/runtimes/linux-x64/native/libtemporal_sdk_bridge.so [0]; destroying link map
1:
1: file=/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libtemporal_sdk_bridge.so [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=/app/libtemporal_sdk_bridge.so [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=libtemporal_sdk_bridge.so [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1: find library=libtemporal_sdk_bridge.so [0]; searching
1: search cache=/etc/ld.so.cache
1: search path=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/lib:/usr/lib (system search path)
1: trying file=/lib/x86_64-linux-gnu/libtemporal_sdk_bridge.so
1: trying file=/usr/lib/x86_64-linux-gnu/libtemporal_sdk_bridge.so
1: trying file=/lib/libtemporal_sdk_bridge.so
1: trying file=/usr/lib/libtemporal_sdk_bridge.so
1:
1:
1: file=/app/runtimes/linux-x64/native/temporal_sdk_bridge [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/temporal_sdk_bridge [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=/app/temporal_sdk_bridge [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=temporal_sdk_bridge [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1: find library=temporal_sdk_bridge [0]; searching
1: search cache=/etc/ld.so.cache
1: search path=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/lib:/usr/lib (system search path)
1: trying file=/lib/x86_64-linux-gnu/temporal_sdk_bridge
1: trying file=/usr/lib/x86_64-linux-gnu/temporal_sdk_bridge
1: trying file=/lib/temporal_sdk_bridge
1: trying file=/usr/lib/temporal_sdk_bridge
1:
1:
1: file=/app/runtimes/linux-x64/native/libtemporal_sdk_bridge [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libtemporal_sdk_bridge [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=/app/libtemporal_sdk_bridge [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1:
1: file=libtemporal_sdk_bridge [0]; dynamically loaded by /usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.15/libcoreclr.so [0]
1: find library=libtemporal_sdk_bridge [0]; searching
1: search cache=/etc/ld.so.cache
1: search path=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/lib:/usr/lib (system search path)
1: trying file=/lib/x86_64-linux-gnu/libtemporal_sdk_bridge
1: trying file=/usr/lib/x86_64-linux-gnu/libtemporal_sdk_bridge
1: trying file=/lib/libtemporal_sdk_bridge
1: trying file=/usr/lib/libtemporal_sdk_bridge
1:
Unhandled exception. System.DllNotFoundException: Unable to load shared library 'temporal_sdk_bridge' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libtemporal_sdk_bridge: cannot open shared object file: No such file or directory
at Temporalio.Bridge.Interop.Methods.runtime_free(Runtime* runtime)
at Temporalio.Bridge.Runtime.ReleaseHandle()
at System.Runtime.InteropServices.SafeHandle.InternalRelease(Boolean disposeOrFinalizeOperation)
at System.Runtime.InteropServices.SafeHandle.Dispose(Boolean disposing)
at System.Runtime.InteropServices.SafeHandle.Finalize()
Any idea @Chad_Retz ?
1 Like
You linked the same link twice. I assume you meant this link for the second one.
We run our CI successfully on multiple forms of Linux. Are you deploying with the Windfows temporal_sdk_bridge.dll on Linux? You need to make libtemporal_sdk_bridge.so available, not temporal_sdk_bridge.dll. I will update the issue.
1 Like
Oh, Excuse me!
No no, we have the libtemporal_sdk_bridge.so file in our runtimes\linux-64\native dir.
Can you provide a simple Dockerfile that fails and the replication steps so I can make sure I am debugging the same thing you’re seeing? You can use the smoke test project if you just need a simple project or just any simple project that instantiates a Temporal client.
Sure @Chad_Retz
I added the Temporal package to the project using the following command on my local machine:
dotnet add package Temporalio --prerelease
So that we have a new dependency in NotificationHelper.csproj file.
And here is our simplified backend dockerfile:
FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS base
RUN apt-get update && apt-get install -y apt-utils libgdiplus libc6-dev && apt-get purge
RUN apt install -y python3-pip python3-cffi python3-brotli python3-pil libpango-1.0-0 libharfbuzz0b libpangoft2-1.0-0 libpango1.0-dev && apt-get purge
RUN pip3 install 'weasyprint==52.5'
RUN mkdir -p /usr/share/fonts/truetype/inter
COPY ["static/Fonts/Inter/*", "/usr/share/fonts/truetype/inter/"]
COPY ["static/Images/.", "/usr/share/Images/"]
RUN chmod 644 /usr/share/fonts/truetype/inter/*
# Rebuild the font cache.
RUN fc-cache -fv
ENV LC_ALL=en_US.UTF-8 \
LANG=en_US.UTF-8
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY ["api/Api/Api.csproj", "api/Api/"]
COPY ["common/NotificationHelper/NotificationHelper.csproj", "common/NotificationHelper/"]
COPY ["nuget.config", "."]
RUN dotnet restore "api/Api/Api.csproj"
RUN dotnet restore "common/NotificationHelper/NotificationHelper.csproj"
COPY . .
WORKDIR "/src/api/Api"
RUN dotnet build "Api.csproj" -c Release -o /app/build
FROM build AS test
WORKDIR /src
RUN mkdir /test_res
RUN dotnet test -c Release --results-directory /test_res --logger:trx
FROM test AS publish
WORKDIR "/src/api/Api"
RUN dotnet publish "Api.csproj" -c Release -o /app/publish
FROM base AS final
RUN apt-get update && apt-get install -y apt-utils libgdiplus libc6-dev && apt purge
RUN mkdir -p /app/wwwroot/upload/f/contents
WORKDIR /app
COPY --from=publish /app/publish .
COPY api/Api/appsettings.sample.json appsettings.json
ENTRYPOINT ["dotnet", "Api.dll"]`
Ideally I could have a Dockerfile I can run that can replicate, but I think I can trim yours up to remove some things. Note, this may take me a week due to some availability issues on my side.
(also note that even when it does load the .so file there is an issue at here where we use too new of a glibc version for that base image you’re using, this will be fixed next release)
1 Like
I set the LB_DEBUG variable and get the following log:
/src/common/NotificationHelper/Service/NotificationSendService.cs:line 75
1: find library=Microsoft.DiaSymReader.Native.arm64.dll [0]; searching
1: search cache=/etc/ld.so.cache
1: search path=/lib/aarch64-linux-gnu:/usr/lib/aarch64-linux-gnu:/lib:/usr/lib (system search path)
1: trying file=/lib/aarch64-linux-gnu/Microsoft.DiaSymReader.Native.arm64.dll
1: trying file=/usr/lib/aarch64-linux-gnu/Microsoft.DiaSymReader.Native.arm64.dll
1: trying file=/lib/Microsoft.DiaSymReader.Native.arm64.dll
1: trying file=/usr/lib/Microsoft.DiaSymReader.Native.arm64.dll
1:
1: find library=Microsoft.DiaSymReader.Native.arm64.dll [0]; searching
1: search cache=/etc/ld.so.cache
1: search path=/lib/aarch64-linux-gnu:/usr/lib/aarch64-linux-gnu:/lib:/usr/lib (system search path)
1: trying file=/lib/aarch64-linux-gnu/Microsoft.DiaSymReader.Native.arm64.dll
1: trying file=/usr/lib/aarch64-linux-gnu/Microsoft.DiaSymReader.Native.arm64.dll
1: trying file=/lib/Microsoft.DiaSymReader.Native.arm64.dll
1: trying file=/usr/lib/Microsoft.DiaSymReader.Native.arm64.dll
@Chad_Retz
This is an old post but I just spend several hours troubleshooting the same issue so I’m writing here in case anyone else has the same problem they might save some hours.
If you’re using an -Alpine image that uses the musl C library it doesn’t work.
The image/server running your application must have glibc like Debian/Ubuntu.
Or you can try install glibc like gcompat but I didn’t manage and gave up and just used a Debian based image instead… And then it worked.
edit: to verify you can run
ldd temporal_sdk_bridge.so
If you in the output see errors and musl its probably easiest to change image.
Yes, we do not natively ship musl-compatible shared library. See this section of the README.
1 Like