Tags: Zipstack/unstract
Tags
[FIX] Frontend CSP: add unsafe-eval for RJSF and blob: for PDF viewer (… …#1875) Fix frontend CSP: add unsafe-eval for RJSF and blob: for PDF viewer - Add 'unsafe-eval' to script-src: RJSF (React JSON Schema Form) uses new Function() to compile schemas, blocked without this directive - Add blob: to connect-src: PDF.js viewer loads documents via blob: URLs which were being blocked, causing "Failed to Load PDF" errors Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
[SECURITY] Add input validation and CSP headers (#1834) * [SECURITY] Add input validation, CSP headers, and secure cookie defaults - Add server-side HTML/script injection validation for name and description fields across all user-facing serializers (CWE-20) - Add Content-Security-Policy header via Django middleware (API) and nginx config (frontend) to mitigate XSS and data injection attacks - Change SESSION_COOKIE_SECURE and CSRF_COOKIE_SECURE defaults to True so cookies are only sent over HTTPS Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Apply suggestion from @coderabbitai[bot] Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Signed-off-by: Hari John Kuriakose <hari@zipstack.com> * Apply suggestion from @coderabbitai[bot] Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Signed-off-by: Hari John Kuriakose <hari@zipstack.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Apply suggestions from code review Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Signed-off-by: Hari John Kuriakose <hari@zipstack.com> * Address PR review comments for input validation and CSP hardening - Narrow EVENT_HANDLER_PATTERN to vetted DOM event names to avoid false positives on benign words like "connection=", "onboarding=", "oncall=" - Add None guards to validate_description in WorkflowSerializer and CustomToolSerializer to prevent TypeError on nullable fields - Add 'unsafe-inline' to script-src and style-src in backend CSP middleware for login.html inline scripts/styles - Add 'object-src none' to backend CSP per OWASP best practice - Remove plaintext ws: from nginx connect-src, keep only wss: - Add 6 regression tests for benign input and non-whitespace delimiters - Update CSP middleware docstring to accurately describe behavior Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Address PR review comments: fix nginx CSP syntax, tighten policies - Fix broken nginx add_header syntax: combine CSP into a single quoted string (nginx does not concatenate adjacent quoted strings) - Tighten img-src from broad https: wildcard to explicit PostHog origins - Replace 'unsafe-inline' with SHA-256 hash for login.html inline script in backend CSP middleware, maintaining strong XSS protection Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Harden input sanitizer: catch unclosed tags and data:/vbscript: URIs - Extend HTML_TAG_PATTERN to catch unclosed tags like "<script" that could be completed by adjacent content in non-React contexts - Extend JS_PROTOCOL_PATTERN to also block data: and vbscript: URIs which can execute scripts when rendered into href/src attributes - Add tests for unclosed tags, data: URI, vbscript:, and "a < 3" benign case Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix data: URI false positives on ordinary English text - Refine JS_PROTOCOL_PATTERN to only match data: when followed by a MIME type (word/word), avoiding false positives on text like "Input data: JSON format" - Add tests for benign "data:" in prose and data: URI with MIME type Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Signed-off-by: Hari John Kuriakose <hari@zipstack.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: vishnuszipstack <vishnu@zipstack.com> Co-authored-by: Kirtiman Mishra <110175055+kirtimanmishrazipstack@users.noreply.github.com>
UN-3211 [FEAT] HTTP session lifecycle management for workers API clie… …nts (#1782) * UN-3211 [FEAT] HTTP session lifecycle management for workers API clients - Add _owns_session flag to prevent singleton shared session from being closed by individual clients - Wire API_CLIENT_POOL_SIZE into HTTPAdapter connection pools - Add idempotent close() and __del__ destructor to BaseAPIClient - Add try/finally cleanup in api-deployment and callback tasks - Add on_worker_process_shutdown hook and early-return guard in postrun - Add 25 unit tests for session lifecycle behavior Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * UN-3211 [FIX] Address CodeRabbit review: log reset failures, thread-safe counter - Log warning instead of silently swallowing exceptions in reset_singleton() - Add threading.Lock around task counter increment for thread safety with threads/gevent/eventlet pools Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * UN-3211 [FIX] Address CodeRabbit review round 2: optimize lock scope, document thread-safety - Move WorkerConfig() instantiation outside lock in increment_task_counter() - Remove redundant _task_counter=0 (already done inside reset_singleton) - Document thread-safety caveat in reset_singleton() docstring - Log close failures in task cleanup instead of silently swallowing Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * UN-3211 [FIX] Address CodeRabbit review round 3: test real on_task_postrun handler Tests now call the real worker.on_task_postrun() signal handler instead of simulating the guard logic inline, catching divergence if the handler's guard, try/except, or import path changes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * UN-3211 [FIX] Optimize singleton reset with cached threshold and improve close() logging Cache the singleton_reset_task_threshold to avoid re-importing WorkerConfig on every task increment. Promote api_client.close() failure logs from debug to warning for better production visibility. Update tests to reset cached threshold. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * UN-3211 [FIX] Improve API execution error handling and clean up worker imports Refactor api-deployment tasks to handle setup failures early with proper cleanup, move shared imports to module level in worker.py, and fix type annotations in client_factory. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Chandrasekharan M <117059509+chandrasekharan-zipstack@users.noreply.github.com> Co-authored-by: Kirtiman Mishra <110175055+kirtimanmishrazipstack@users.noreply.github.com>
UN-3318 [FEAT] Platform API key post-merge fixes (#1863) * UN-3318 [FEAT] Platform API key implementation - Change API keys from user-scoped to org-scoped (all admins see all keys) - Add IsOrganizationAdmin permission — only admins can manage keys - Create dedicated service account (is_service_account) per key for bearer auth - Auto-add key owner as shared_users co-owner on resources created via API key - Filter service accounts from member listings and shared_users serializers - Admin-gate frontend route and nav item for Platform API Keys - Add "Created By" column to API keys table * [FIX] Add optional chaining for socket event handlers in SocketMessages * [FIX] Add platform_api URLs to main URL configuration * UN-3318 [FEAT] Platform API key enhancements and prompt sync endpoint - Add ownership transfer on key deletion (service account -> key creator) - Fix service account email to not expose key ID (use name-slug format) - Add retrieve endpoint for fetching full API key (copy-to-clipboard) - Revert service account filtering from serializers (show in user lists) - Add POST /prompt-studio/<id>/sync-prompts/ for cross-env prompt sync - Fix export_tool crash when user_id not provided (set(None) error) - Update copy button UX: icon after key text, fetch real key from backend - Fix worker script shebang for macOS bash compatibility * UN-3318 [FIX] Move platform API keys route outside RequireAdmin gate * UN-3318 [FIX] Address PR review feedback - Return 401 instead of 500 for unconfigured API key api_user - Wrap get_response in try/finally for StateStore cleanup - Add max_length=512 to description CharField in serializer - Include forward slash in validation error message - Fix useEffect deps for fetchKeys (guard on orgId, add fetchKeys dep) - Remove redundant ModuleNotFoundError from except clause * UN-3318 [FIX] Add pre_delete signal for service account cleanup Move service account cleanup from view to pre_delete signal so it fires on all deletion paths (admin, cascade, scripts), not just the API view. * UN-3318 [FIX] Simplify service account email to key-name@platform.internal Use key name directly for email without UUID suffix. Username retains UUID prefix for global uniqueness. Cross-org isolation is handled by the org-scoped manager mixin on OrganizationMember. * UN-3318 [FEAT] Add read/read_write permission on API keys and bypass owner checks for service accounts - Add permission field (read/read_write) to PlatformApiKey model - Enforce in middleware: block DELETE, block writes for read-only keys - Bypass for_user owner/shared checks for service accounts (all 7 managers) - Bypass object-level permissions (IsOwner, IsOwnerOrSharedUser, etc.) - Hide service accounts from member listings and shared_users serializers - Add permission column, select in create/edit modals on frontend * UN-3318 [FIX] Merge implicitly concatenated string in db_comment * UN-3318 [FIX] Regenerate platform_api migrations (squash permission into initial) * Revert unintended shebang change in workers/run-worker.sh * Fix route gating: move platform-api-keys inside RequireAdmin, restore triad position * Add UniqueTogetherValidator for platform API key name+org uniqueness DRF doesn't auto-validate UniqueConstraint from Meta.constraints (unlike unique_together), so duplicate names would cause a 500 IntegrityError. Adding explicit validator returns a clean 400. * Fix name uniqueness validation to use manual check instead of UniqueTogetherValidator UniqueTogetherValidator requires all fields in serializer data, but organization is set at model layer via DefaultOrganizationMixin.save(). Use validate_name with UserContext.get_organization() instead. * Use accessible Button element for API key copy action Replace clickable div with Ant Design Button (type=text) for keyboard accessibility and proper ARIA semantics. * Fix Greptile review issues: atomicity, email collision, model scan scope - Wrap create_api_user_for_key in transaction.atomic() to prevent orphaned service accounts on partial failure - Wrap sync_prompts delete+recreate in transaction.atomic() to prevent data loss if import_prompts fails after delete - Add uid fragment to service account email to prevent collision across orgs with same key name - Scope transfer_ownership to business app labels only instead of scanning all Django models (avoids system/third-party model queries) - Fix frontend SAFE_TEXT_MESSAGE to include forward slashes * Fix code review issues: cross-org leak, signal safety, email sanitization - WorkflowExecutionManager: scope service account queryset by org via workflow__organization to prevent cross-org data leak - pre_delete signal: wrap in try/except so cleanup failures don't block key deletion (orphaned accounts logged instead) - Email sanitization: strip non-alphanumeric chars from key name before using in email to comply with RFC 5321 - Move uuid/re imports to top of services.py - Add exception logging to IsOrganizationAdmin permission class * Address human reviewer feedback: middleware, serializer, error messages - Move DELETE check before DB lookup in bearer auth middleware - Improve error message for missing service account (actionable) - Include max count in KeyCountExceeded error message - Move key count validation from view to serializer - Add TODO for key hashing on model - Remove unused imports from views.py * Remove platform API key count limit and related constants/exceptions The limit was an arbitrary safety net attracting unnecessary review discussion. No rate limiting needed at this stage. * Reduce cognitive complexity in transfer_ownership to satisfy SonarCloud Extract per-model logic into _transfer_model_ownership helper to reduce nesting depth and bring cognitive complexity from 21 to within the allowed threshold of 15. * Use serializer for sync_prompts validation, remove manual error handling Add SyncPromptsSerializer to validate request body with DictField and required keys check. Remove try/except blocks from sync_prompts view to let DRF middleware handle errors consistently. * Add type hints to services.py and use constants for HTTP methods - Add type annotations to all public functions in platform_api/services.py - Add RequestMethod constants class with SAFE_METHODS frozenset - Use RequestMethod and ApiKeyPermission enum in bearer auth middleware instead of hardcoded strings * Address code review: transactions, retrieve masking, permission hardening - Wrap create() in transaction.atomic() so key + service account are atomic (no orphaned keys if service account creation fails) - Wrap transfer_ownership and delete_api_user_for_key in transaction.atomic() to prevent partial ownership transfers - Use masked serializer for retrieve() — plaintext key only shown at create and rotate - Use save(update_fields=) in rotate() for efficiency - Use _meta.get_fields() instead of hasattr() for Django field detection - Narrow service account permission bypass to SAFE_METHODS only in IsOwner, IsOwnerOrSharedUser, IsOwnerOrSharedUserOrSharedToOrg * Fix handleCopyKey to use masked value from list data The retrieve endpoint now returns masked keys, so the copy button should use the already-available masked value from the record instead of making a redundant API call. * Revert retrieve masking — full key needed for copy-to-clipboard The retrieve endpoint must return the full key since handleCopyKey relies on it. Key hashing (show plaintext only at create/rotate) should be done as a separate cross-cutting change with APIDeployment. * Fix NoneType error when created_by is NULL in prompt studio serializer Handle NULL created_by in CustomToolSerializer.to_representation to prevent AttributeError on projects where the creator has been deleted. * Make API key name immutable after creation Name is used to derive the service account username/email, so it should not be editable. Remove name from update serializer and show it as disabled in the edit modal. * Fix ownership transfer on API key deletion and prompt studio bugs - Use _base_manager to bypass DefaultOrganizationManagerMixin in transfer_ownership (org context unavailable during signal paths) - Dynamically discover all User FK/M2M fields instead of hardcoding created_by/modified_by (fixes missed workflow_owner transfer) - Move created_by_email assignment before early return in prompt studio serializer so new projects without prompts show the owner - Add trailing slash to prompt studio share PATCH URL * Fix workflow_manager app_label in transfer ownership scan The app_label for the Workflow model is "workflow_v2" (derived from AppConfig.name = "workflow_manager.workflow_v2"), not "workflow_manager". This caused transfer_ownership to skip workflows entirely, leaving created_by/workflow_owner NULL after service account deletion. * Allow service account write access and auto-share imported tools - Broaden service account permission bypass from SAFE_METHODS only to all non-DELETE methods (DELETE already blocked at middleware layer) - Auto-add API key owner to shared_users when service account creates tools via import/sync backup, so they appear in the admin's UI * Add logger.warning for missing PlatformApiKey during tool import * Update backend/permissions/permission.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> Signed-off-by: Chandrasekharan M <117059509+chandrasekharan-zipstack@users.noreply.github.com> * Improve observability: log warning when created_by is null, expand docstring * Fix UserSessionUtils.get_organization_id for Bearer token requests When authenticated via Platform API Key (Bearer token), there is no session, so get_organization_id returned None. This caused TypeError in file upload and other endpoints that build paths using org_id. Fall back to request.organization_id (set by OrganizationMiddleware from the URL) when a platform_api_key is present on the request. * Fix stale session edge case in UserSessionUtils, use user.id in logs - Unconditionally return URL org_id for Bearer token requests, ignoring any stale session cookie that may also be present - Use user.id instead of user.username in warning logs to avoid PII --------- Signed-off-by: Chandrasekharan M <117059509+chandrasekharan-zipstack@users.noreply.github.com> Co-authored-by: Chandrasekharan M <117059509+chandrasekharan-zipstack@users.noreply.github.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
UN-3318 [FEAT] Platform API key implementation (#1860) * UN-3318 [FEAT] Platform API key implementation - Change API keys from user-scoped to org-scoped (all admins see all keys) - Add IsOrganizationAdmin permission — only admins can manage keys - Create dedicated service account (is_service_account) per key for bearer auth - Auto-add key owner as shared_users co-owner on resources created via API key - Filter service accounts from member listings and shared_users serializers - Admin-gate frontend route and nav item for Platform API Keys - Add "Created By" column to API keys table * [FIX] Add optional chaining for socket event handlers in SocketMessages * [FIX] Add platform_api URLs to main URL configuration * UN-3318 [FEAT] Platform API key enhancements and prompt sync endpoint - Add ownership transfer on key deletion (service account -> key creator) - Fix service account email to not expose key ID (use name-slug format) - Add retrieve endpoint for fetching full API key (copy-to-clipboard) - Revert service account filtering from serializers (show in user lists) - Add POST /prompt-studio/<id>/sync-prompts/ for cross-env prompt sync - Fix export_tool crash when user_id not provided (set(None) error) - Update copy button UX: icon after key text, fetch real key from backend - Fix worker script shebang for macOS bash compatibility * UN-3318 [FIX] Move platform API keys route outside RequireAdmin gate * UN-3318 [FIX] Address PR review feedback - Return 401 instead of 500 for unconfigured API key api_user - Wrap get_response in try/finally for StateStore cleanup - Add max_length=512 to description CharField in serializer - Include forward slash in validation error message - Fix useEffect deps for fetchKeys (guard on orgId, add fetchKeys dep) - Remove redundant ModuleNotFoundError from except clause * UN-3318 [FIX] Add pre_delete signal for service account cleanup Move service account cleanup from view to pre_delete signal so it fires on all deletion paths (admin, cascade, scripts), not just the API view. * UN-3318 [FIX] Simplify service account email to key-name@platform.internal Use key name directly for email without UUID suffix. Username retains UUID prefix for global uniqueness. Cross-org isolation is handled by the org-scoped manager mixin on OrganizationMember. * UN-3318 [FEAT] Add read/read_write permission on API keys and bypass owner checks for service accounts - Add permission field (read/read_write) to PlatformApiKey model - Enforce in middleware: block DELETE, block writes for read-only keys - Bypass for_user owner/shared checks for service accounts (all 7 managers) - Bypass object-level permissions (IsOwner, IsOwnerOrSharedUser, etc.) - Hide service accounts from member listings and shared_users serializers - Add permission column, select in create/edit modals on frontend * UN-3318 [FIX] Merge implicitly concatenated string in db_comment * UN-3318 [FIX] Regenerate platform_api migrations (squash permission into initial) * Revert unintended shebang change in workers/run-worker.sh * Fix route gating: move platform-api-keys inside RequireAdmin, restore triad position * Add UniqueTogetherValidator for platform API key name+org uniqueness DRF doesn't auto-validate UniqueConstraint from Meta.constraints (unlike unique_together), so duplicate names would cause a 500 IntegrityError. Adding explicit validator returns a clean 400. * Fix name uniqueness validation to use manual check instead of UniqueTogetherValidator UniqueTogetherValidator requires all fields in serializer data, but organization is set at model layer via DefaultOrganizationMixin.save(). Use validate_name with UserContext.get_organization() instead. * Use accessible Button element for API key copy action Replace clickable div with Ant Design Button (type=text) for keyboard accessibility and proper ARIA semantics. * Fix Greptile review issues: atomicity, email collision, model scan scope - Wrap create_api_user_for_key in transaction.atomic() to prevent orphaned service accounts on partial failure - Wrap sync_prompts delete+recreate in transaction.atomic() to prevent data loss if import_prompts fails after delete - Add uid fragment to service account email to prevent collision across orgs with same key name - Scope transfer_ownership to business app labels only instead of scanning all Django models (avoids system/third-party model queries) - Fix frontend SAFE_TEXT_MESSAGE to include forward slashes * Fix code review issues: cross-org leak, signal safety, email sanitization - WorkflowExecutionManager: scope service account queryset by org via workflow__organization to prevent cross-org data leak - pre_delete signal: wrap in try/except so cleanup failures don't block key deletion (orphaned accounts logged instead) - Email sanitization: strip non-alphanumeric chars from key name before using in email to comply with RFC 5321 - Move uuid/re imports to top of services.py - Add exception logging to IsOrganizationAdmin permission class * Address human reviewer feedback: middleware, serializer, error messages - Move DELETE check before DB lookup in bearer auth middleware - Improve error message for missing service account (actionable) - Include max count in KeyCountExceeded error message - Move key count validation from view to serializer - Add TODO for key hashing on model - Remove unused imports from views.py * Remove platform API key count limit and related constants/exceptions The limit was an arbitrary safety net attracting unnecessary review discussion. No rate limiting needed at this stage. * Reduce cognitive complexity in transfer_ownership to satisfy SonarCloud Extract per-model logic into _transfer_model_ownership helper to reduce nesting depth and bring cognitive complexity from 21 to within the allowed threshold of 15. * Use serializer for sync_prompts validation, remove manual error handling Add SyncPromptsSerializer to validate request body with DictField and required keys check. Remove try/except blocks from sync_prompts view to let DRF middleware handle errors consistently. * Add type hints to services.py and use constants for HTTP methods - Add type annotations to all public functions in platform_api/services.py - Add RequestMethod constants class with SAFE_METHODS frozenset - Use RequestMethod and ApiKeyPermission enum in bearer auth middleware instead of hardcoded strings * Address code review: transactions, retrieve masking, permission hardening - Wrap create() in transaction.atomic() so key + service account are atomic (no orphaned keys if service account creation fails) - Wrap transfer_ownership and delete_api_user_for_key in transaction.atomic() to prevent partial ownership transfers - Use masked serializer for retrieve() — plaintext key only shown at create and rotate - Use save(update_fields=) in rotate() for efficiency - Use _meta.get_fields() instead of hasattr() for Django field detection - Narrow service account permission bypass to SAFE_METHODS only in IsOwner, IsOwnerOrSharedUser, IsOwnerOrSharedUserOrSharedToOrg * Fix handleCopyKey to use masked value from list data The retrieve endpoint now returns masked keys, so the copy button should use the already-available masked value from the record instead of making a redundant API call. * Revert retrieve masking — full key needed for copy-to-clipboard The retrieve endpoint must return the full key since handleCopyKey relies on it. Key hashing (show plaintext only at create/rotate) should be done as a separate cross-cutting change with APIDeployment. * Fix NoneType error when created_by is NULL in prompt studio serializer Handle NULL created_by in CustomToolSerializer.to_representation to prevent AttributeError on projects where the creator has been deleted. * Make API key name immutable after creation Name is used to derive the service account username/email, so it should not be editable. Remove name from update serializer and show it as disabled in the edit modal. * Fix ownership transfer on API key deletion and prompt studio bugs - Use _base_manager to bypass DefaultOrganizationManagerMixin in transfer_ownership (org context unavailable during signal paths) - Dynamically discover all User FK/M2M fields instead of hardcoding created_by/modified_by (fixes missed workflow_owner transfer) - Move created_by_email assignment before early return in prompt studio serializer so new projects without prompts show the owner - Add trailing slash to prompt studio share PATCH URL * Fix workflow_manager app_label in transfer ownership scan The app_label for the Workflow model is "workflow_v2" (derived from AppConfig.name = "workflow_manager.workflow_v2"), not "workflow_manager". This caused transfer_ownership to skip workflows entirely, leaving created_by/workflow_owner NULL after service account deletion.
UN-3334 [FIX] Fix profile dropdown menu items alignment (#1861) * UN-3334 [FIX] Fix profile dropdown menu items center-aligned instead of left-aligned Ant Design 5.13 Button type="text" introduced justify-content: center, overriding the default flex-start alignment in the profile dropdown menu. Add explicit justify-content: flex-start to .logout-button and ensure plugin-rendered menu items are also left-aligned via .ant-dropdown-menu-title-content. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add flex-start to menu-title-content for plugin flex items Address review feedback: text-align alone won't override justify-content on flex children. Making the wrapper itself a flex container with justify-content: flex-start ensures plugin items using antd Button are also left-aligned. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * UN-3334 [FIX] Fix biome CSS formatting for TopNavBar selector Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: vishnuszipstack <vishnu@zipstack.com>
[FIX] Redis Sentinel support for tool containers and sidecars (#1858) * [FIX] Fix Redis Sentinel crash in MetricsMixin (prompt-service) MetricsMixin was creating a raw StrictRedis client using REDIS_HOST/PORT directly, which in Sentinel HA mode points to the Sentinel process (port 26379). Sentinel doesn't support the SELECT command, causing ResponseError on every /answer-prompt call. - Replace StrictRedis with create_redis_client() from unstract-core which handles Sentinel discovery via master_for() - Add unstract-core as sdk1 dependency (all services using sdk1 already depend on core, so zero new transitive deps) - Add error handling around set_start_time() and collect_metrics() so Redis failures degrade gracefully instead of crashing requests - Add missing COPY for unstract/core in tool Dockerfiles (classifier, structure, text_extractor) since sdk1 now depends on core Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Commit uv.lock changes * [FIX] Add unstract-core to tool requirements.txt for pip resolution Tool Dockerfiles (classifier, text_extractor, structure) use pip, not uv. Since sdk1 now depends on unstract-core (which isn't on PyPI), pip can't resolve it via [tool.uv.sources]. Adding explicit editable install of core before sdk1 ensures pip finds it locally. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * [REFACTOR] DRY Redis client creation in worker cache modules Worker cache modules (cache_backends.py, cache_utils.py) duplicated sentinel/standalone branching logic for Redis client creation. Both now use create_redis_client() from unstract-core as the single entry point, which already handles mode detection internally. - Add SSL support (ssl, ssl_cert_reqs) to create_redis_client() via env vars ({prefix}SSL, {prefix}SSL_CERT_REQS), disabled by default - Remove if-sentinel-else-standalone branching in cache_backends.py and cache_utils.py — replaced with single create_redis_client() call - Remove unused os and redis imports from simplified modules Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * [FIX] Pass Redis Sentinel env vars to tool and sidecar containers The runner was not forwarding REDIS_SENTINEL_MODE and REDIS_SENTINEL_MASTER_NAME to tool containers and sidecars, causing the sidecar to connect in standalone mode to the Sentinel port (26379) and fail on all data commands (HGETALL, SETEX). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * [FIX] Bump tool versions for Redis Sentinel compatibility (#1859) Patch bump for tools that depend on unstract-core's updated Redis client with Sentinel support: - classifier: 0.0.76 -> 0.0.77 - structure: 0.0.97 -> 0.0.98 - text_extractor: 0.0.72 -> 0.0.73 Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
[FIX] Fix Redis Sentinel crash in MetricsMixin (#1856) * [FIX] Fix Redis Sentinel crash in MetricsMixin (prompt-service) MetricsMixin was creating a raw StrictRedis client using REDIS_HOST/PORT directly, which in Sentinel HA mode points to the Sentinel process (port 26379). Sentinel doesn't support the SELECT command, causing ResponseError on every /answer-prompt call. - Replace StrictRedis with create_redis_client() from unstract-core which handles Sentinel discovery via master_for() - Add unstract-core as sdk1 dependency (all services using sdk1 already depend on core, so zero new transitive deps) - Add error handling around set_start_time() and collect_metrics() so Redis failures degrade gracefully instead of crashing requests - Add missing COPY for unstract/core in tool Dockerfiles (classifier, structure, text_extractor) since sdk1 now depends on core Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Commit uv.lock changes * [FIX] Add unstract-core to tool requirements.txt for pip resolution Tool Dockerfiles (classifier, text_extractor, structure) use pip, not uv. Since sdk1 now depends on unstract-core (which isn't on PyPI), pip can't resolve it via [tool.uv.sources]. Adding explicit editable install of core before sdk1 ensures pip finds it locally. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * [REFACTOR] DRY Redis client creation in worker cache modules (#1857) Worker cache modules (cache_backends.py, cache_utils.py) duplicated sentinel/standalone branching logic for Redis client creation. Both now use create_redis_client() from unstract-core as the single entry point, which already handles mode detection internally. - Add SSL support (ssl, ssl_cert_reqs) to create_redis_client() via env vars ({prefix}SSL, {prefix}SSL_CERT_REQS), disabled by default - Remove if-sentinel-else-standalone branching in cache_backends.py and cache_utils.py — replaced with single create_redis_client() call - Remove unused os and redis imports from simplified modules Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PreviousNext