Failed Jobs

Enqueued 0 / 0 Scheduled 0 Processing 0 Succeeded 61,989 Failed 5 Deleted 0 Awaiting 0

Failed jobs do not become expired to allow you to re-queue them without any time pressure. You should re-queue or delete them manually, or apply AutomaticRetry(OnAttemptsExceeded = AttemptsExceededAction.Delete) attribute to delete them automatically.

Id	Job	Failed
#31949	ISendEmailJob.ProcessPendingEmailsAsync Failed to change state to a 'Processing' one due to an exception after 10 retry attempts Fewer details...	11/25/2025 15:39:23
#31949	System.InvalidOperationException (lw0mdlwk0001l5:1875) Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached. System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached. at Microsoft.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection) at Microsoft.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions) at Microsoft.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry, SqlConnectionOverrides overrides) at Microsoft.Data.SqlClient.SqlConnection.Open(SqlConnectionOverrides overrides) at Hangfire.SqlServer.SqlServerStorage.CreateAndOpenConnection() in C:\projects\hangfire-525\src\Hangfire.SqlServer\SqlServerStorage.cs:line 406 at Hangfire.SqlServer.SqlServerConnection.AcquireLock(String resource, TimeSpan timeout) in C:\projects\hangfire-525\src\Hangfire.SqlServer\SqlServerConnection.cs:line 800 at Hangfire.SqlServer.SqlServerConnection.AcquireDistributedLock(String resource, TimeSpan timeout) in C:\projects\hangfire-525\src\Hangfire.SqlServer\SqlServerConnection.cs:line 67 at Hangfire.Storage.StorageConnectionExtensions.AcquireDistributedJobLock(IStorageConnection connection, String jobId, TimeSpan timeout) in C:\projects\hangfire-525\src\Hangfire.Core\Storage\StorageConnectionExtensions.cs:line 33 at Hangfire.States.BackgroundJobStateChanger.ChangeState(StateChangeContext context) in C:\projects\hangfire-525\src\Hangfire.Core\States\BackgroundJobStateChanger.cs:line 69 at Hangfire.Server.Worker.TryChangeState(BackgroundProcessContext context, IStorageConnection connection, String jobId, IState state, IReadOnlyDictionary`2 customData, String[] expectedStates, IFetchedJob completeJob, BackgroundJob& backgroundJob, CancellationToken initializeToken, CancellationToken abortToken) in C:\projects\hangfire-525\src\Hangfire.Core\Server\Worker.cs:line 235
#20436	AsyncTaskProcessorJob.ProcessPendingTasks An exception occurred during performance of the job. More details...	11/21/2025 15:35:15
#20436	System.ArgumentOutOfRangeException (lw0mdlwk0001l5:1875) Token 2000000 is not valid in the scope of module System.ModuleHandle. (Parameter 'typeToken') System.ArgumentOutOfRangeException: Token 2000000 is not valid in the scope of module System.ModuleHandle. (Parameter 'typeToken') at System.ModuleHandle.ResolveTypeHandle(Int32 typeToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext) at System.Reflection.RuntimeModule.ResolveType(Int32 metadataToken, Type[] genericTypeArguments, Type[] genericMethodArguments) at System.Reflection.CustomAttribute.AddCustomAttributes(ListBuilder`1& attributes, RuntimeModule decoratedModule, Int32 decoratedMetadataToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder`1 derivedAttributes) at Microsoft.EntityFrameworkCore.Internal.DbContextServices.<CreateModel>g__FindCompiledModel\|6_0(Type contextType) at Microsoft.EntityFrameworkCore.Internal.DbContextServices.CreateModel(Boolean designTime) at Microsoft.EntityFrameworkCore.Infrastructure.EntityFrameworkServicesBuilder.<>c.<TryAddCoreServices>b__8_4(IServiceProvider p) at ResolveService(ILEmitResolverBuilderRuntimeContext, ServiceProviderEngineScope) at ResolveService(ILEmitResolverBuilderRuntimeContext, ServiceProviderEngineScope) at ResolveService(ILEmitResolverBuilderRuntimeContext, ServiceProviderEngineScope) at ResolveService(ILEmitResolverBuilderRuntimeContext, ServiceProviderEngineScope) at ResolveService(ILEmitResolverBuilderRuntimeContext, ServiceProviderEngineScope) at ResolveService(ILEmitResolverBuilderRuntimeContext, ServiceProviderEngineScope) at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType) at Microsoft.EntityFrameworkCore.DbContext.get_DbContextDependencies() at Microsoft.EntityFrameworkCore.DbContext.get_ContextServices() at Microsoft.EntityFrameworkCore.DbContext.get_DbContextDependencies() at Microsoft.EntityFrameworkCore.DbContext.Set[TEntity]() at PGMS.DataProvider.EFCore.Services.UnitOfWork`1.GetDbSet[TEntity]() at PGMS.DataProvider.EFCore.Services.BaseOperationEntityRepository`1.GetOperationQuery[TEntity](IUnitOfWork unitOfWork, Expression`1 filter) at PGMS.DataProvider.EFCore.Services.BaseOperationEntityRepository`1.GetOperationAsync[TEntity](IUnitOfWork unitOfWork, Expression`1 filter, Func`2 orderBy, Int32 fetchSize, Int32 offset) at PGMS.DataProvider.EFCore.Services.BaseEntityRepository`1.FindTopAsync[TEntity](Expression`1 filter, Func`2 orderBy, Int32 fetchSize, Int32 offset) at PGMS.DataProvider.EFCore.Services.BaseEntityRepository`1.FindTopAsync[TEntity](Expression`1 filter, Func`2 orderBy, Int32 fetchSize, Int32 offset) at TEQ.WorkerRole.Services.AsyncTaskProcessorService.ProcessPendingTasks() in D:\a\1\s\src\TEQ.WorkerRole\Services\AsyncTaskProcessorService.cs:line 30 at TEQ.WorkerRole.Jobs.AsyncTaskProcessorJob.ProcessPendingTasks() in D:\a\1\s\src\TEQ.WorkerRole\Jobs\AsyncTaskProcessorJob.cs:line 27 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
#20434	AsyncTaskProcessorJob.ProcessPendingTasks An exception occurred during performance of the job. More details...	11/21/2025 15:34:05
#20434	System.ArgumentOutOfRangeException (lw0mdlwk0001l5:1875) Token 2000000 is not valid in the scope of module System.ModuleHandle. (Parameter 'typeToken') System.ArgumentOutOfRangeException: Token 2000000 is not valid in the scope of module System.ModuleHandle. (Parameter 'typeToken') at System.ModuleHandle.ResolveTypeHandle(Int32 typeToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext) at System.Reflection.RuntimeModule.ResolveType(Int32 metadataToken, Type[] genericTypeArguments, Type[] genericMethodArguments) at System.Reflection.CustomAttribute.AddCustomAttributes(ListBuilder`1& attributes, RuntimeModule decoratedModule, Int32 decoratedMetadataToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder`1 derivedAttributes) at Microsoft.EntityFrameworkCore.Internal.DbContextServices.<CreateModel>g__FindCompiledModel\|6_0(Type contextType) at Microsoft.EntityFrameworkCore.Internal.DbContextServices.CreateModel(Boolean designTime) at Microsoft.EntityFrameworkCore.Infrastructure.EntityFrameworkServicesBuilder.<>c.<TryAddCoreServices>b__8_4(IServiceProvider p) at ResolveService(ILEmitResolverBuilderRuntimeContext, ServiceProviderEngineScope) at ResolveService(ILEmitResolverBuilderRuntimeContext, ServiceProviderEngineScope) at ResolveService(ILEmitResolverBuilderRuntimeContext, ServiceProviderEngineScope) at ResolveService(ILEmitResolverBuilderRuntimeContext, ServiceProviderEngineScope) at ResolveService(ILEmitResolverBuilderRuntimeContext, ServiceProviderEngineScope) at ResolveService(ILEmitResolverBuilderRuntimeContext, ServiceProviderEngineScope) at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType) at Microsoft.EntityFrameworkCore.DbContext.get_DbContextDependencies() at Microsoft.EntityFrameworkCore.DbContext.get_ContextServices() at Microsoft.EntityFrameworkCore.DbContext.get_DbContextDependencies() at Microsoft.EntityFrameworkCore.DbContext.Set[TEntity]() at PGMS.DataProvider.EFCore.Services.UnitOfWork`1.GetDbSet[TEntity]() at PGMS.DataProvider.EFCore.Services.BaseOperationEntityRepository`1.GetOperationQuery[TEntity](IUnitOfWork unitOfWork, Expression`1 filter) at PGMS.DataProvider.EFCore.Services.BaseOperationEntityRepository`1.GetOperationAsync[TEntity](IUnitOfWork unitOfWork, Expression`1 filter, Func`2 orderBy, Int32 fetchSize, Int32 offset) at PGMS.DataProvider.EFCore.Services.BaseEntityRepository`1.FindTopAsync[TEntity](Expression`1 filter, Func`2 orderBy, Int32 fetchSize, Int32 offset) at PGMS.DataProvider.EFCore.Services.BaseEntityRepository`1.FindTopAsync[TEntity](Expression`1 filter, Func`2 orderBy, Int32 fetchSize, Int32 offset) at TEQ.WorkerRole.Services.AsyncTaskProcessorService.ProcessPendingTasks() in D:\a\1\s\src\TEQ.WorkerRole\Services\AsyncTaskProcessorService.cs:line 30 at TEQ.WorkerRole.Jobs.AsyncTaskProcessorJob.ProcessPendingTasks() in D:\a\1\s\src\TEQ.WorkerRole\Jobs\AsyncTaskProcessorJob.cs:line 27 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
#17887	AsyncTaskProcessorJob.ProcessPendingTasks An exception occurred during performance of the job. More details...	11/20/2025 18:20:06
#17887	System.ArgumentException (lw0mdlwk0001l5:1874) A BadImageFormatException has been thrown while parsing the signature. This is likely due to lack of a generic context. Ensure genericTypeArguments and genericMethodArguments are provided and contain enough context. System.ArgumentException: A BadImageFormatException has been thrown while parsing the signature. This is likely due to lack of a generic context. Ensure genericTypeArguments and genericMethodArguments are provided and contain enough context. ---> System.BadImageFormatException: The signature is incorrect. The format of the file '/home/site/wwwroot/TEQ.Data.dll' is invalid. at System.ModuleHandle.ResolveType(QCallModule module, Int32 typeToken, IntPtr* typeInstArgs, Int32 typeInstCount, IntPtr* methodInstArgs, Int32 methodInstCount, ObjectHandleOnStack type) at System.ModuleHandle.ResolveTypeHandle(Int32 typeToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext) at System.Reflection.RuntimeModule.ResolveType(Int32 metadataToken, Type[] genericTypeArguments, Type[] genericMethodArguments) --- End of inner exception stack trace --- at System.Reflection.RuntimeModule.ResolveType(Int32 metadataToken, Type[] genericTypeArguments, Type[] genericMethodArguments) at System.Attribute.InternalIsDefined(PropertyInfo element, Type attributeType, Boolean inherit) at System.Attribute.IsDefined(MemberInfo element, Type attributeType) at PGMS.DataProvider.EFCore.Services.BaseOperationEntityRepository`1.HasLazyLoading(Type type) at PGMS.DataProvider.EFCore.Services.BaseOperationEntityRepository`1.GetOperationQuery[TEntity](IUnitOfWork unitOfWork, Expression`1 filter) at PGMS.DataProvider.EFCore.Services.BaseOperationEntityRepository`1.GetOperationAsync[TEntity](IUnitOfWork unitOfWork, Expression`1 filter, Func`2 orderBy, Int32 fetchSize, Int32 offset) at PGMS.DataProvider.EFCore.Services.BaseEntityRepository`1.FindTopAsync[TEntity](Expression`1 filter, Func`2 orderBy, Int32 fetchSize, Int32 offset) at PGMS.DataProvider.EFCore.Services.BaseEntityRepository`1.FindTopAsync[TEntity](Expression`1 filter, Func`2 orderBy, Int32 fetchSize, Int32 offset) at TEQ.WorkerRole.Services.AsyncTaskProcessorService.ProcessPendingTasks() in D:\a\1\s\src\TEQ.WorkerRole\Services\AsyncTaskProcessorService.cs:line 30 at TEQ.WorkerRole.Jobs.AsyncTaskProcessorJob.ProcessPendingTasks() in D:\a\1\s\src\TEQ.WorkerRole\Jobs\AsyncTaskProcessorJob.cs:line 27 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
#17611	ConvertAccountTask.Execute An exception occurred during performance of the job. More details...	11/20/2025 17:44:06
#17611	System.Exception (lw0mdlwk0001l5:1874) Incomplete run - we encounter some problems converting users System.Exception: Incomplete run - we encounter some problems converting users at TEQ.WorkerRole.Jobs.Conversions.ConvertAccountTask.ExecuteTask(PerformContext context, CancellationToken cancellationToken) in D:\a\1\s\src\TEQ.WorkerRole\Jobs\Conversions\ConvertAccountTask.cs:line 74 at TEQ.WorkerRole.Jobs.BaseBackgroundTask.Execute(PerformContext context, CancellationToken cancellationToken) in D:\a\1\s\src\TEQ.WorkerRole\Jobs\BaseBackgroundTask.cs:line 37 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

System.InvalidOperationException (lw0mdlwk0001l5:1875)

System.ArgumentOutOfRangeException (lw0mdlwk0001l5:1875)

System.ArgumentOutOfRangeException (lw0mdlwk0001l5:1875)

System.ArgumentException (lw0mdlwk0001l5:1874)

System.Exception (lw0mdlwk0001l5:1874)