Failed Jobs

Enqueued 0 / 0 Scheduled 0 Processing 0 Succeeded 197,303 Failed 5,862 Deleted 0 Awaiting 0

Failed jobs do not become expired to allow you to re-queue them without any time pressure. You should re-queue or delete them manually, or apply AutomaticRetry(OnAttemptsExceeded = AttemptsExceededAction.Delete) attribute to delete them automatically.

Id	Job	Failed
#108661	ExecuteBackgroundJobTask.StartExecuteBackgroundJob An exception occurred during performance of the job. Fewer details...	11/25/2025 01:39:58
#108661	System.Net.Http.HttpRequestException Response status code does not indicate success: 502 (Bad Gateway). `System.Net.Http.HttpRequestException: Response status code does not indicate success: 502 (Bad Gateway). at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() at SMP.Schedules.ExecuteBackgroundJobTask.StartExecuteBackgroundJob() in /home/azureuser/smp/src/BackEnd/SMP.Schedules/Tasks/ExecuteBackgroundJobTask.cs:line 25 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)`
#108660	SendNotificationsTask.StartJobSendNotifications An exception occurred during performance of the job. More details...	11/25/2025 01:39:58
#108660	System.Net.Http.HttpRequestException Response status code does not indicate success: 502 (Bad Gateway). `System.Net.Http.HttpRequestException: Response status code does not indicate success: 502 (Bad Gateway). at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() at SMP.Schedules.SendNotificationsTask.StartJobSendNotifications() in /home/azureuser/smp/src/BackEnd/SMP.Schedules/Tasks/SendNotificationsTask.cs:line 34 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)`
#108658	SendNotificationsTask.StartJobSendNotifications An exception occurred during performance of the job. More details...	11/25/2025 01:39:57
#108658	System.Threading.Tasks.TaskCanceledException The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing. System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing. ---> System.TimeoutException: The operation was canceled. ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled. ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled. ---> System.Net.Sockets.SocketException (125): Operation canceled --- End of inner exception stack trace --- at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token) at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](CancellationToken cancellationToken, Int32 estimatedSize) at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token) at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](Memory`1 buffer, CancellationToken cancellationToken) at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token) at System.Net.Http.HttpConnection.InitialFillAsync(Boolean async) at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpClient.<SendAsync>g__Core\|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) --- End of inner exception stack trace --- --- End of inner exception stack trace --- at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts) at System.Net.Http.HttpClient.<SendAsync>g__Core\|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) at SMP.Schedules.SendNotificationsTask.StartJobSendNotifications() in /home/azureuser/smp/src/BackEnd/SMP.Schedules/Tasks/SendNotificationsTask.cs:line 33 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#108651	ExecuteBackgroundJobTask.StartExecuteBackgroundJob An exception occurred during performance of the job. More details...	11/25/2025 01:34:46
#108651	System.Net.Http.HttpRequestException Response status code does not indicate success: 502 (Bad Gateway). `System.Net.Http.HttpRequestException: Response status code does not indicate success: 502 (Bad Gateway). at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() at SMP.Schedules.ExecuteBackgroundJobTask.StartExecuteBackgroundJob() in /home/azureuser/smp/src/BackEnd/SMP.Schedules/Tasks/ExecuteBackgroundJobTask.cs:line 25 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)`
#108650	SendNotificationsTask.StartJobSendNotifications An exception occurred during performance of the job. More details...	11/25/2025 01:34:46
#108650	System.Net.Http.HttpRequestException Response status code does not indicate success: 502 (Bad Gateway). `System.Net.Http.HttpRequestException: Response status code does not indicate success: 502 (Bad Gateway). at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() at SMP.Schedules.SendNotificationsTask.StartJobSendNotifications() in /home/azureuser/smp/src/BackEnd/SMP.Schedules/Tasks/SendNotificationsTask.cs:line 34 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)`
#108648	SendNotificationsTask.StartJobSendNotifications An exception occurred during performance of the job. More details...	11/25/2025 01:34:42
#108648	System.Threading.Tasks.TaskCanceledException The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing. System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing. ---> System.TimeoutException: The operation was canceled. ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled. at System.Net.Http.CancellationHelper.ThrowOperationCanceledException(Exception innerException, CancellationToken cancellationToken) at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpClient.<SendAsync>g__Core\|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) --- End of inner exception stack trace --- --- End of inner exception stack trace --- at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts) at System.Net.Http.HttpClient.<SendAsync>g__Core\|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) at SMP.Schedules.SendNotificationsTask.StartJobSendNotifications() in /home/azureuser/smp/src/BackEnd/SMP.Schedules/Tasks/SendNotificationsTask.cs:line 33 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#108634	SendNotificationsTask.StartJobSendNotifications An exception occurred during performance of the job. More details...	11/25/2025 01:27:54
#108634	System.Net.Http.HttpRequestException Connection refused (testing.sourcenowmarketplace.com:443) System.Net.Http.HttpRequestException: Connection refused (testing.sourcenowmarketplace.com:443) ---> System.Net.Sockets.SocketException (111): Connection refused at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation\|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.InjectNewHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpClient.<SendAsync>g__Core\|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) at SMP.Schedules.SendNotificationsTask.StartJobSendNotifications() in /home/azureuser/smp/src/BackEnd/SMP.Schedules/Tasks/SendNotificationsTask.cs:line 33 at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#108619	SendNotificationsTask.StartJobSendNotifications An exception occurred during performance of the job. More details...	11/24/2025 21:48:25
#108619	System.Net.Http.HttpRequestException Connection refused (testing.sourcenowmarketplace.com:443) System.Net.Http.HttpRequestException: Connection refused (testing.sourcenowmarketplace.com:443) ---> System.Net.Sockets.SocketException (111): Connection refused at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation\|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.InjectNewHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpClient.<SendAsync>g__Core\|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) at SMP.Schedules.SendNotificationsTask.StartJobSendNotifications() in /home/azureuser/smp/src/BackEnd/SMP.Schedules/Tasks/SendNotificationsTask.cs:line 33 at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#108612	ExecuteBackgroundJobTask.StartExecuteBackgroundJob An exception occurred during performance of the job. More details...	11/24/2025 21:02:45
#108612	System.Net.Http.HttpRequestException Response status code does not indicate success: 502 (Bad Gateway). `System.Net.Http.HttpRequestException: Response status code does not indicate success: 502 (Bad Gateway). at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() at SMP.Schedules.ExecuteBackgroundJobTask.StartExecuteBackgroundJob() in /home/azureuser/smp/src/BackEnd/SMP.Schedules/Tasks/ExecuteBackgroundJobTask.cs:line 25 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)`
#108611	SendNotificationsTask.StartJobSendNotifications An exception occurred during performance of the job. More details...	11/24/2025 21:02:45
#108611	System.Net.Http.HttpRequestException Response status code does not indicate success: 502 (Bad Gateway). `System.Net.Http.HttpRequestException: Response status code does not indicate success: 502 (Bad Gateway). at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() at SMP.Schedules.SendNotificationsTask.StartJobSendNotifications() in /home/azureuser/smp/src/BackEnd/SMP.Schedules/Tasks/SendNotificationsTask.cs:line 34 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)`

System.Net.Http.HttpRequestException

System.Net.Http.HttpRequestException

System.Threading.Tasks.TaskCanceledException

System.Net.Http.HttpRequestException

System.Net.Http.HttpRequestException

System.Threading.Tasks.TaskCanceledException

System.Net.Http.HttpRequestException

System.Net.Http.HttpRequestException

System.Net.Http.HttpRequestException

System.Net.Http.HttpRequestException