8249 Commits

Author SHA1 Message Date
ViacheslavKlimov
bb7ea08e59 EDQS: refactoring, OOM handling, healthcheck 2025-02-14 12:08:05 +02:00
dashevchenko
f128f49d44 test fixes 2025-02-05 18:57:23 +02:00
dashevchenko
09bd3de984 test fixes 2025-02-05 16:42:31 +02:00
dashevchenko
b2bf5b9f65 configurable edqs sync batch sizes 2025-02-05 12:20:23 +02:00
dashevchenko
60736e1237 fixed entity removal from CustomerData 2025-02-04 12:23:35 +02:00
dashevchenko
0e9d0b3ba3 fixed EntityServiceTest 2025-02-03 18:27:36 +02:00
ViacheslavKlimov
c8db304ce6 Optimized postgres sync (#3208)
* optimized postgres sync

* Fix consumer stopping when stopWhenRead enabled

* Fix NPE on EDQS repartitioning

* fixed EntityServiceTest

* Fix EDQS yml props

---------

Co-authored-by: ViacheslavKlimov <viacheslavklimov11@gmail.com>
2025-01-30 12:43:14 +02:00
Viacheslav Klimov
86b5378d59 EDQS (#3196)
* Experiments with CSV

* CSV Loader v1

* EDQ tests

* Volatile variables instead of final

* Improvements

* updated loader with new entities

* Fix double memory usage issue

* Basic data structures and load

* Minor improvements

* Snappy + Large String reuse

* added EntityFields classes for each entity

* Basic implementation

* Minor improvements to KeyFilters

* implemented RepositoryUtils.checkKeyFilters

* Generic query implementation

* New structure

* Refactoring and few processors implementation

* extended DeviceData with shared/client attributes and device profile

* Minor refactoring of attribute scopes

* DeviceTypeFilter support

* Strong types of fields for each entity data class

* DeviceType and AssetType filters

* EntityView and Edge queries

* Relations Query

* Relation Query Implementation

* Update EDQS module version

* Sync with EDQS via Kafka

* EDQS: major refactoring

* EDQS API requests via Kafka

* EDQS: full sync with the database

* Refactoring for EDQS sync

* EDQS: major refactoring and new features

* EDQS refactoring, count query support, fix tests

* EDQS: refactoring for query processors

* Fix EDQS pom version

* Cleanup edqs.yml

* EDQS: tenant partitioning strategy; refactoring

* EDQS: latest events queue

* EDQS: support for monolith setup; RocksDB; other improvements

* EDQS: merge sync and events topics, introduce state topic

* EDQS: dynamic repartitioning

* implemented entity data query filters for edqs

* EdqsEntityQueryControllerTest - use in-memory queue

* edqs-filter fixes, added test

* EDQS: blob entity support

* EdqsEntityQueryControllerTest - use in-memory queue

* Use DummyEdqsService when disabled

* Fixes for EDQS

* Refactoring for EDQS tests

* Fix edqs requests partitioning

* EDQS: Fix for attributes handling

* Fix attributes saving in EntityServiceTest

* EDQS: refactoring, fixes

* Minor refactoring for query processor

* added ownerName/ownerType support

* fixed relation query processor

* fixed EntityServiceTest

* refactoring

* added support for parentId for relation query result

* Get rid of EntityNameFetcher

* Add fixme for relation query processor

* db restore with select all edqs fields

* fixed entity deletion

* fixed FieldUtils with new EntityFields

* dao method renamed

* EDQS: instance groups with same partitions; automatic sync; multiple fixes

* Refactoring for EDQS sync

* EDQS: refactoring

* Fix startup with Kafka

* fixed EntityQueryControllerTest

* fixed EdqsEntityServiceTest

* Separate queue admin for EDQS request template

* Implement new EDQS partitioning strategy

* EDQS: multiple fixes and refactoring

* Add mock EdqsRocksDb beans to tests

* added edqs stats for inmemory/grafana

* fixed filter tests

* Update todos

* Refactoring for QueueConfig

* Improvements and refactoring for EDQS consumers

* implemented TODOs

* test fixes

* Consume state topic up to end offsets

* edqs stats refactoring

* EDQS: cleanup on partitions removal; refactoring

* EDQS: minor refactoring

* EDQS: remove CSV loader

---------

Co-authored-by: Andrii Shvaika <ashvayka@thingsboard.io>
Co-authored-by: dashevchenko <dshevchenko@thingsboard.io>
2025-01-27 17:41:12 +02:00
Igor Kulikov
f920010786 Merge branch 'rc' 2025-01-22 18:25:11 +02:00
yevhenii_zahrebelnyi
9d873cb0bf
Fix telemetry edge test (#12496)
* Fix telemetry edge test

- Changed random failure logic for DownlinkMsg. A DownlinkMsg can fail only twice.

* - add case with failureProbability 100

* - refactored naming
2025-01-22 15:45:39 +02:00
Viacheslav Klimov
f4b5c4321b
Merge pull request #12398 from thingsboard/fix_bug_lwm2m_profile_update_dynamic
fix bug: lwm2m update Observe dynamically and between connections
2025-01-14 17:38:25 +02:00
Igor Kulikov
03bc07b528
Merge pull request #12372 from ArtemDzhereleiko/AD/imp/scada-trd/meters
Add Meter traditional SCADA symbols
2025-01-10 18:59:55 +02:00
Igor Kulikov
7c8d945cb9
Merge pull request #12274 from ArtemDzhereleiko/AD/imp/oil-gas-scada-bundle
HP SCADA Oil and gas symbols with bundle
2025-01-10 18:59:18 +02:00
Andrew Shvayka
801c06aea0
Merge pull request #12405 from thingsboard/tbel_new_bytes_parsing_methods
tbel: add hexToBytesArray, base64ToBytesList
2025-01-10 16:48:17 +02:00
nick
80e93d8a40 Merge branch 'master' into fix_bug_lwm2m_profile_update_dynamic 2025-01-09 17:44:47 +02:00
nick
3821e91573 Merge branch 'master' into fix/coap-dtls-session 2025-01-09 10:37:07 +02:00
nick
df8d9e2540 Merge branch 'master' into fix_bug_lwm2m_profile_update_dynamic 2025-01-08 16:17:56 +02:00
nick
d961b89f31 tbel: add hexToBytesArray, base64ToBytesList 2025-01-08 16:17:14 +02:00
nick
c82659bd95 fix bug test: lwm2m profile without params 2025-01-08 11:50:36 +02:00
nick
e5df59456a fix bug: update Observe dynamically and between connections due to profile changes 2025-01-07 18:40:19 +02:00
Viacheslav Klimov
c4925bd8af
Merge pull request #12366 from smatvienko-tb/feature/tbel-test-refactor
tbel test refactoring
2025-01-07 10:40:26 +02:00
Kulikov
c5ca395844
fix_bug_coaps_x509_docker_many_devices (#12327)
* coaps: x509 - tests

* coaps: x509 - tests add sever cert.pem

* coaps: x509 - tests add sever cert.pem -2

* coaps: x509 - tests add sever cert.pem -3

* coaps: x509 - tests add sever cert.pem -4

* fix bug: coaps x509 - ddocker many devices with re-port

* fix bug: coaps - add test, connect client with X509

* fix bug: coaps - add two test devices with the same port

* fix bug: coaps - add two test devices with the same port (FeatureType.ATTRIBUTES)

* fix bug: coaps - add two test devices with the same port (FeatureType.ATTRIBUTES) - 1

* fix bug: coap comments 1
2025-01-03 18:17:30 +02:00
Sergey Matvienko
e1b29b5004 removed custom reflection methods in favour ReflectionUtils 2025-01-02 15:21:28 +01:00
Artem Dzhereleiko
6a5cc07742 UI: Traditional SCADA symbols meter 2025-01-02 13:32:17 +02:00
Artem Dzhereleiko
1d0c998af8 UI: Add serch tag for drilling 2025-01-02 12:58:28 +02:00
Viacheslav Klimov
0a1f377347
Merge pull request #12199 from thingsboard/develop/4.0
develop/4.0
2025-01-02 10:47:58 +02:00
Sergey Matvienko
527f7176c5 tbel test refactored to not use sql and all controllers. reflection utils used 2025-01-01 14:25:28 +01:00
Igor Kulikov
1a8116cb93 Merge with rc 2024-12-31 11:52:06 +02:00
Andrew Shvayka
45c6b063bf
Merge pull request #12203 from thingsboard/tbel_doc_bitWiseOperations
tbel_tests_doc_bitwise_operations
2024-12-31 10:12:48 +02:00
ViacheslavKlimov
6c9b0de0bc Fix CaffeineCacheDefaultConfigurationTest 2024-12-31 10:08:43 +02:00
Volodymyr Babak
f35147e146 Temporary disabled relatedEdges due to issue with scan method 2024-12-30 18:37:59 +02:00
yevhenii_zahrebelnyi
c0f4440bd9
Relation for Edge (#12353)
* Relation for Edge

- Removed the relation filtering for Edge.

* Relation for Edge

- add test for relation Edge

* Relation for Edge

- refactoring

* Relation for Edge

- Naming fixed
2024-12-30 17:46:03 +02:00
Viacheslav Klimov
d9f37cd2f2
Merge pull request #12360 from thingsboard/fix/upgrade-from
Ignore version from tb_schema_settings when from_version property is specified
2024-12-30 17:41:41 +02:00
ViacheslavKlimov
aba14c7ecc Remove invalid upgrade log 2024-12-30 16:51:27 +02:00
ViacheslavKlimov
7c6b3f6adf Temporal: ignore version in tb_schema_settings when from_version property is specified 2024-12-30 16:45:29 +02:00
Viacheslav Klimov
949adf0ccb
Merge pull request #12358 from AndriiLandiak/fix/edge-flaky
Fix test: testRuleChains
2024-12-30 16:24:59 +02:00
Viacheslav Klimov
21fef65229
Merge pull request #12359 from thingsboard/fix/system-notification-delivery
Ignore unconfigured delivery method for system notification type
2024-12-30 16:24:43 +02:00
ViacheslavKlimov
f02b25f15a Ignore unconfigured delivery method for system notification type 2024-12-30 16:15:37 +02:00
Andrii Landiak
2f8aa65a8e Fix RuleChain flaky test 2024-12-30 16:14:01 +02:00
nick
ffd1d891cd fix bug: lwm2m flaky test (.times to .atLeast) 2024-12-30 12:15:08 +02:00
Viacheslav Klimov
64c687c5a9
Merge branch 'master' into develop/4.0 2024-12-30 12:03:36 +02:00
Volodymyr Babak
cf9204416b
Fixed case when seqId started new cycle. Improved edge trace/debug logging (#12342)
* Fixed case when seqId started new cycle. Improved edge trace/debug logging

* Edge downlink logging improved

* Edge downlink logging improved #2
2024-12-30 10:04:41 +02:00
Igor Kulikov
15d22e9e6b Improve upgrade script to separate long running resources update. 2024-12-27 17:15:59 +02:00
Igor Kulikov
3a2228918b Fix schema upgrade script. 2024-12-27 16:23:17 +02:00
Igor Kulikov
94f294aebf Fix load of JS Module extensions during installation or upgrade. 2024-12-27 15:32:17 +02:00
Igor Kulikov
d2293388ed
Merge pull request #12315 from maxunbearable/improvement/gateway-offline-backups
Added offline backups for Gateways
2024-12-27 12:04:00 +02:00
mpetrov
30ce285725 Updated JS module 2024-12-26 16:19:14 +02:00
Igor Kulikov
36f990cf15 Merge branch 'rc' 2024-12-24 14:20:44 +02:00
Igor Kulikov
23cd86e870
Merge pull request #12331 from ArtemDzhereleiko/AD/hot-fix/scada-scale-invert
Fixed SCADA scale symbols
2024-12-24 14:19:27 +02:00
Artem Dzhereleiko
4c7775eba9 UI: Fixed scale symbol 2024-12-24 12:56:50 +02:00