This introduces a new Raw Data Export module to the codebase, enabling users to export raw log data via a dedicated API endpoint. The changes include the implementation of the module and handler, integration with existing infrastructure, configuration updates, and adjustments to tests and module wiring.
## 📄 Summary
To reliably migrate the alerts and dashboards, we need access to the telemetrystore to fetch some metadata and while doing migration, I need to log some stuff to fix stuff later.
Key changes:
- Modified the migration to include telemetrystore and a logging provider (open to using a standard logger instead)
- To avoid the previous issues with imported dashboards failing during migration, I've ensured that imported JSON files are automatically transformed when migration is active
- Implemented detailed logic to handle dashboard migration cleanly and prevent unnecessary errors
- Separated the core migration logic from SQL migration code, as users from the dot metrics migration requested shareable code snippets for local migrations. This modular approach allows others to easily reuse the migration functionality.
Known: I didn't register the migration yet in this PR, and will not merge this yet, so please review with that in mid.
Deprecate all flags
- Use querier.config.fluxInterval in lieu of passing `--flux-interval` and `--flux-interval-for-trace-detail`
- Remove `--gateway-url`
- Use telemetrystore.clickhouse.cluster in lieu of passing `--cluster` or `--cluster-name`
- Add an `unparam` check in the linter. Updated some functions across the querier codebase to be compatible with this linter.
- Remove prometheus config from docker builds.
* feat(user): support sso and api key
* feat(user): remove ee references from pkg
* feat(user): remove ee references from pkg
* feat(user): related client changes
* feat(user): remove the sso available check
* feat(user): fix go tests
* feat(user): move the middleware from ee to pkg
* feat(user): some more error code cleanup
* feat(user): some more error code cleanup
* feat(user): skip flaky UI tests
* feat(user): some more error code cleanup
* chore(linter): add more linters and deprecate zap
* chore(linter): add more linters and deprecate zap
* chore(linter): add more linters and deprecate zap
* chore(linter): add more linters and deprecate zap
* chore(savedview): refactor into module and handler
* chore(rule): move telemetry inside telemetry
* chore(apdex): refactor apdex and delete dao
* chore(dashboard): create a dashboard module
* chore(ee): get rid of the init nonesense
* chore(dashboard): fix err and apierror confusion
* chore: address comments
* feat: added changes related to custom options for quick filters
* feat: added changes related to custom options for quick filters
* feat: added changes related to custom options for quick filters
* feat: added changes related to custom options for quick filters
* feat: added changes related to custom options for quick filters
* feat: added changes related to custom options for quick filters
* feat: added support for custom quick filters
* feat: added changes related to custom options for quick filters
* feat: added changes related to custom options for quick filters
* feat: added changes related to custom options for quick filters
* feat: added changes related to custom options for quick filters
* feat: added changes related to custom options for quick filters
* feat: added changes related to custom options for quick filters
* feat: added changes related to custom options for quick filters
* feat: added changes related to custom options for quick filters
* feat(organization): add hname and alias for organization
* fix: boolean values are not shown in the list panel's column
* fix: moved logic to component level
* fix: added type
* fix: added test cases
* fix: added test cases
* chore: update copy webpack plugin
* Revert "fix: display same key with multiple data types in filter suggestions by enhancing the deduping logic (#7255)"
This reverts commit 1e85981a17a8e715e948308d3e85072d976907d3.
* fix: use query search v2 for traces data source to handle multiple data types for the same key
* fix(QueryBuilderSearchV2): add user typed option if it doesn't exist in the payload
* fix(QueryBuilderSearchV2): increase the height of search dropdown for non-logs data sources
* fix: display span scope selector for trace data source
* chore: remove the span scope selector from qb search v1 and move the component to search v2
* fix: write test to ensure that we display span scope selector for traces data source
* fix: limit converting -> only to log data source
* fix: don't display empty suggestion if only spaces are typed
* chore: tests for span scope selector
* chore: qb search flow (key, operator, value) test cases
* refactor: fix the Maximum update depth reached issue while running tests
* chore: overall improvements to span scope selector tests
Resource attr filter: style fix and quick filter changes (#7691)
* chore: resource attr filter init
* chore: resource attr filter api integration
* chore: operator config updated
* chore: fliter show hide logic and styles
* chore: add support for custom operator list to qb
* chore: minor refactor
* chore: minor code refactor
* test: quick filters test suite added
* test: quick filters test suite added
* test: all errors test suite added
* chore: style fix
* test: all errors mock fix
* chore: test case fix and mixpanel update
* chore: color update
* chore: minor refactor
* chore: style fix
* chore: set default query in exceptions tab
* chore: style fix
* chore: minor refactor
* chore: minor refactor
* chore: minor refactor
* chore: test update
* chore: fix filter header with no query name
* fix: scroll fix
* chore: add data source traces to quick filters
* chore: replace div with fragment
---------
Co-authored-by: Aditya Singh <adityasingh@Adityas-MacBook-Pro.local>
fix: handle rate operators for table panel (#7695)
* fix: handle rate operators for table panel
chore: fix error rate (#7701)
Signed-off-by: Shivanshu Raj Shrivastava <shivanshu1333@gmail.com>
* feat(organization): minor cleanups
* feat(organization): better naming for api and usecase
* feat(organization): better packaging for modules
* feat(organization): change hname to displayName
* feat(organization): update the migration to use dialect
* feat(organization): update the migration to use dialect
* feat(organization): update the migration to use dialect
* feat(organization): revert back to impl
* feat(organization): remove DI from organization
* feat(organization): address review comments
* feat(organization): address review comments
* feat(organization): address review comments
---------
Signed-off-by: Shivanshu Raj Shrivastava <shivanshu1333@gmail.com>
* chore: multitenancy in integrations
* chore: multitenancy in cloud integration accounts
* chore: changes to cloudintegrationservice
* chore: rename migration
* chore: update scan function
* chore: update scan function
* chore: fix migration
* chore: fix struct
* chore: remove unwanted code
* chore: update scan function
* chore: migrate user and pat for integrations
* fix: changes to the user for integrations
* fix: address comments
* fix: copy created_at
* fix: update non revoked token
* chore: don't allow deleting pat and user for integrations
* fix: address comments
* chore: address comments
* chore: add checks for fk in dialect
* fix: service migration
* fix: don't update user if user is already migrated
* fix: update correct service config
* fix: remove unwanted code
* fix: remove migration for multiple same services which is not required
* fix: fix migration and disable disaboard if metrics disabled
* fix: don't use ee types
---------
Co-authored-by: Vikrant Gupta <vikrant@signoz.io>
* feat(sqlmigration): update the alertmanager tables
* feat(sqlmigration): update the alertmanager tables
* feat(sqlmigration): make the preference package multi tenant
* feat(preference): address nit pick comments
* feat(preference): added the cascade delete for preferences
* feat(sqlmigration): update apdex and TTL status tables (#7481)
* feat(sqlmigration): update the apdex and ttl tables
* feat(sqlmigration): register the new migration and rename table
* feat(sqlmigration): fix the ttl queries
* feat(sqlmigration): update the TTL and apdex tables
* feat(sqlmigration): update the TTL and apdex tables
* fix: support multitenancy in org
* fix: register and login working now
* fix: changes to migration
* fix: migrations run both on sqlite and postgres
* fix: remove user flags from fe and be
* fix: remove ingestion keys from update
* fix: multitenancy support for apdex settings
* fix: render ts for users correctly
* fix: fix migration to run for new tenants
* fix: clean up migrations
* fix: address comments
* Update pkg/sqlmigration/013_update_organization.go
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* fix: fix build
* fix: force invites with org id
* Update pkg/query-service/auth/auth.go
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* fix: address comments
* fix: address comments
* fix: provier with their own dialect
* fix: update dialects
* fix: remove unwanted change
* Update pkg/query-service/app/http_handler.go
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* fix: different files for types
---------
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* chore: include CWAgent namespace for ec2 metrics
* chore: update aws integration service metrics connection status queried labels
* chore: fix breaking test
* chore: cloudintegrations: rename rds def
* feat: cloudintegrations: shape of agent check in response for v0 release
* chore: cloudintegrations: add validation for response expected after agent check in
* chore: cloudintegrations: accumulate teletry collection strategies for enabled services
* chore: cloudintegrations: use map struct to parse from map to struct with json tags
* chore: cloudintegrations: telemetry collection strategy for services
* chore: cloudintegrations: wrap up test for agent check in resp
* chore: some cleanup
* chore: some cleanup
* chore: some minor renaming
* chore: address review comment
* chore: some cleanup
---------
Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
* feat: cloud service integrations: get model and repo interface started
* feat: cloud service integrations: flesh out more of cloud services model
* feat: cloud integrations: reorganize things a little
* feat: cloud integrations: get svc controller started
* feat: cloud integrations: add stubs for EC2 and RDS postgres services
* feat: cloud integrations: add validation for listing and getting available svcs and some cleanup
* feat: cloud integrations: refactor helpers in existing integrations code for reuse
* feat: cloud integrations: parsing of cloud service definitions
* feat: cloud integrations: impl for getCloudProviderService
* feat: cloud integrations: some reorganization
* feat: cloud integrations: some more cleanup
* feat: cloud integrations: add validation for listing available cloud provider services
* feat: cloud integrations: API endpoint for listing available cloud provider services
* feat: cloud integrations: add validation for getting details of a particular service
* feat: cloud integrations: API endpoint for getting details of a service
* feat: cloud integrations: add controller validation for configuring cloud services
* feat: cloud integrations: get serviceConfigRepo started
* feat: cloud integrations: service config in service list summaries when queried for cloud account id
* feat: cloud integrations: only a supported service for a connected cloud account can be configured
* feat: cloud integrations: add validation for configuring services via the API
* feat: cloud integrations: API for configuring services
* feat: cloud integrations: some cleanup
* feat: cloud integrations: fix broken test
---------
Co-authored-by: Srikanth Chekuri <srikanth.chekuri92@gmail.com>
* feat: init app/cloud_integrations
* feat: get API test started for cloudintegrations account lifecycle
* feat: cloudintegrations: get controller started
* feat: cloud integrations: add cloudintegrations.Controller to APIHandler and servers
* feat: cloud integrations: get routes started
* feat: cloud integrations: get accounts table schema started
* feat: cloud integrations: get cloudProviderAccountsSQLRepository started
* feat: cloud integrations: cloudProviderAccountsSQLRepository.listAccounts
* feat: cloud integrations: http handler and controller plumbing for /generate-connection-url
* feat: cloud integrations: cloudProviderAccountsSQLRepository.upsert
* feat: cloud integrations: finish up with /generate-connection-url
* feat: cloud integrations: add cloudProviderAccountsRepository.get
* feat: cloud integrations: add API test expectation for being able to get account status
* feat: cloud integrations: add http handler and controller method for getting account status
* feat: cloud integrations: ensure unconnected accounts aren't included in list of connected accounts
* feat: cloud integrations: add test expectation for agent check in request
* feat: cloud integrations: agent check in API
* feat: cloud integrations: ensure polling for status after agent check in works
* feat: cloud integrations: ensure account included in connected account list after agent check in
* feat: cloud integrations: add API expectation for updating account config
* feat: cloud integrations: API for updating cloud account config
* feat: cloud integrations: expectation for agent receiving latest config after account config update
* feat: cloud integrations: expectation for disconnecting cloud accounts from UI
* feat: cloud integrations: API for disconnecting cloud accounts
* feat: cloud integrations: some cleanup
* feat: cloud integrations: some more cleanup
* feat: cloud integrations: repo: scope rows by cloud provider
* feat: testutils: refactor out helper for creating a test sqlite DB
* feat: cloud integrations: controller: add test validating regeneration of connection url
* feat: cloud integrations: controller: validations for agent check ins
* feat: cloud integrations: connected account response structure
* feat: cloud integrations: API response account structure
* feat: cloud integrations: some more cleanup
* feat: cloud integrations: remove cloudProviderAccountsRepository.GetById
* feat: cloud integrations: shouldn't be able to disconnect non-existent account
* feat: cloud integrations: validate agents can't check in to cloud account with 2 signoz ids
* feat: cloud integrations: ensure agents can't check in to cloud account with 2 signoz ids
* feat: cloud integrations: remove stray import of ee/model in cloudintegrations controller
