152 Commits

Author SHA1 Message Date
Volodymyr Babak
cc5d9dbb2c Added sync of fat edge consumer group into per edge group. Fixed clean up edge topics 2025-04-11 12:44:09 +03:00
ViacheslavKlimov
b40fa86bac Refactoring and fixes for CF lifecycle events handling 2025-03-13 17:35:00 +02:00
ViacheslavKlimov
43f9083f70 Merge branch 'master' of github.com:thingsboard/thingsboard into edqs 2025-02-28 17:12:33 +02:00
ViacheslavKlimov
44439584f4 Merge branch 'master' of github.com:thingsboard/thingsboard into feature/calculated-fields 2025-02-25 11:49:56 +02:00
ViacheslavKlimov
e7ed98910a Merge branch 'master' of github.com:thingsboard/thingsboard into edqs 2025-02-25 11:10:31 +02:00
Igor Kulikov
5cf26d4851 Update license header 2025-02-25 09:39:16 +02:00
Andrii Shvaika
83b338c697 Entity lifecycle implementation 2025-01-31 13:54:53 +02:00
Andrii Shvaika
5aabbd0f1e Link dispatch implementation 2025-01-29 15:37:36 +02:00
Viacheslav Klimov
86b5378d59 EDQS (#3196)
* Experiments with CSV

* CSV Loader v1

* EDQ tests

* Volatile variables instead of final

* Improvements

* updated loader with new entities

* Fix double memory usage issue

* Basic data structures and load

* Minor improvements

* Snappy + Large String reuse

* added EntityFields classes for each entity

* Basic implementation

* Minor improvements to KeyFilters

* implemented RepositoryUtils.checkKeyFilters

* Generic query implementation

* New structure

* Refactoring and few processors implementation

* extended DeviceData with shared/client attributes and device profile

* Minor refactoring of attribute scopes

* DeviceTypeFilter support

* Strong types of fields for each entity data class

* DeviceType and AssetType filters

* EntityView and Edge queries

* Relations Query

* Relation Query Implementation

* Update EDQS module version

* Sync with EDQS via Kafka

* EDQS: major refactoring

* EDQS API requests via Kafka

* EDQS: full sync with the database

* Refactoring for EDQS sync

* EDQS: major refactoring and new features

* EDQS refactoring, count query support, fix tests

* EDQS: refactoring for query processors

* Fix EDQS pom version

* Cleanup edqs.yml

* EDQS: tenant partitioning strategy; refactoring

* EDQS: latest events queue

* EDQS: support for monolith setup; RocksDB; other improvements

* EDQS: merge sync and events topics, introduce state topic

* EDQS: dynamic repartitioning

* implemented entity data query filters for edqs

* EdqsEntityQueryControllerTest - use in-memory queue

* edqs-filter fixes, added test

* EDQS: blob entity support

* EdqsEntityQueryControllerTest - use in-memory queue

* Use DummyEdqsService when disabled

* Fixes for EDQS

* Refactoring for EDQS tests

* Fix edqs requests partitioning

* EDQS: Fix for attributes handling

* Fix attributes saving in EntityServiceTest

* EDQS: refactoring, fixes

* Minor refactoring for query processor

* added ownerName/ownerType support

* fixed relation query processor

* fixed EntityServiceTest

* refactoring

* added support for parentId for relation query result

* Get rid of EntityNameFetcher

* Add fixme for relation query processor

* db restore with select all edqs fields

* fixed entity deletion

* fixed FieldUtils with new EntityFields

* dao method renamed

* EDQS: instance groups with same partitions; automatic sync; multiple fixes

* Refactoring for EDQS sync

* EDQS: refactoring

* Fix startup with Kafka

* fixed EntityQueryControllerTest

* fixed EdqsEntityServiceTest

* Separate queue admin for EDQS request template

* Implement new EDQS partitioning strategy

* EDQS: multiple fixes and refactoring

* Add mock EdqsRocksDb beans to tests

* added edqs stats for inmemory/grafana

* fixed filter tests

* Update todos

* Refactoring for QueueConfig

* Improvements and refactoring for EDQS consumers

* implemented TODOs

* test fixes

* Consume state topic up to end offsets

* edqs stats refactoring

* EDQS: cleanup on partitions removal; refactoring

* EDQS: minor refactoring

* EDQS: remove CSV loader

---------

Co-authored-by: Andrii Shvaika <ashvayka@thingsboard.io>
Co-authored-by: dashevchenko <dshevchenko@thingsboard.io>
2025-01-27 17:41:12 +02:00
IrynaMatveieva
0e1cd69e34 updated cf consumer 2025-01-27 15:55:40 +02:00
IrynaMatveieva
a652b31d7f implementation processing notification 2025-01-24 15:41:00 +02:00
IrynaMatveieva
5641626443 added processNotification impl 2025-01-24 10:53:13 +02:00
Andrii Shvaika
4c71b9d5f6 WIP: Cluster mode implementation 2025-01-23 13:56:44 +02:00
IrynaMatveieva
2080b439a7 added entity added/deleted events handling 2024-11-28 11:28:51 +02:00
IrynaMatveieva
31103e90e3 added implementation to handle update entity profiles events 2024-11-22 17:17:59 +02:00
IrynaMatveieva
3072861a8f implemented methods for calculated field update/delete in cluster service 2024-11-13 17:42:55 +02:00
Andrii Landiak
5602807dbe
Introduce Edge queue to handle edge notification messages 2024-08-29 12:51:20 +03:00
Andrii Landiak
c6ee0bff19
Add edgeIdsCache to decrease number of calls to db for findRelatedEdgeByEntityId 2024-08-27 12:41:39 +03:00
IrynaMatveieva
74ac5a949f added rule engine controller and send rest api call reply node 2024-05-14 10:05:41 +03:00
ViacheslavKlimov
55dc23b9ff Merge with hotfix/3.6.2 2024-02-09 11:38:40 +02:00
ViacheslavKlimov
a571153b7c Single update message for multiple queues 2024-01-24 18:08:01 +02:00
Andrii Shvaika
a3471c367a Merge PR#9968 2024-01-16 11:35:31 +02:00
Igor Kulikov
c5a72ed8df Update license header to 2024 year. 2024-01-09 10:46:16 +02:00
Volodymyr Babak
d329df0323 Merge remote-tracking branch 'upstream/master' into feature/broadcast-logs-improvement 2024-01-02 19:37:39 +02:00
dashevchenko
ba2e877a08 refactoring: moved proto files and converter classes to separate module - proto 2023-11-30 13:06:44 +02:00
Andrew Shvayka
db3e5ba105
Merge pull request #9616 from YevhenBondarenko/fix/prod-2445
Fixed assign device to tenant
2023-11-29 16:40:42 +02:00
YevhenBondarenko
80fc2d9b86 improvements 2023-11-16 15:06:00 +01:00
Andrii Shvaika
a4bac1e794 Implement cache for the entity creation request 2023-11-15 15:57:58 +02:00
YevhenBondarenko
316a09854c implemented device actor delete msg and fixed assign to tenant 2023-11-12 22:15:57 +01:00
Volodymyr Babak
6358b0be90 Admin settings update. @author AndriiLandiak 2023-11-09 18:16:40 +02:00
Andrii Shvaika
c7874aa7e1 Merge branch 'hotfix/3.6' 2023-11-01 12:52:39 +02:00
Andrii Shvaika
268b9cc468 Reserve failed event 2023-11-01 12:35:20 +02:00
Andrii Shvaika
659ebc0340 Reserve deactivated event 2023-11-01 12:32:20 +02:00
Andrii Shvaika
d2cda21d25 Merge hotfix branch 2023-11-01 10:24:37 +02:00
Andrew Shvayka
3ec2e0956f
Merge pull request #9185 from AndriiLandiak/feature/edge-sync-improvement
Edge Synchronization improvement using EdgeId
2023-10-31 17:54:28 +02:00
Andrii Landiak
ddfac584a0 Make componentLifecycle bytes deprecated. Cleaning code with a single line for TbClusterService 2023-10-31 09:58:41 +02:00
dashevchenko
86f8965eab fixed thingsboard.yml 2023-10-30 12:21:30 +02:00
dashevchenko
62703d3583 merged with upstream 2023-10-30 12:19:24 +02:00
Andrii Shvaika
b4fa0b81ab Deprecated some old structures 2023-10-27 15:36:59 +03:00
Andrii Shvaika
e20523b502 Attribute scope check improvement 2023-10-26 19:02:56 +03:00
Andrii Shvaika
d78bb26a73 Merge with hotfix/3.6 2023-10-26 17:11:29 +03:00
Andrii Shvaika
694b6bf851 Refactoring: new data structures and message flow
Local Subscription Service refactoring

Refactoring of the proto structures

Bug fixing after refactoring

Minor improvements and bug fixes

Improvements to avoid additional DB calls

Bug fixes after refactoring

Cluster mode improvements
2023-10-26 17:07:16 +03:00
Andrii Landiak
be4c408324 Merge remote-tracking branch 'upstream/hotfix/3.6' into feature/core-proto-subtype-implementation 2023-10-25 16:30:43 +03:00
Andrew Shvayka
5c404649f3
Merge pull request #9453 from thingsboard/re-consumer-service-refactoring
Rule Engine consumer service refactoring
2023-10-24 13:24:19 +03:00
Andrii Shvaika
b2098fc112 Fix compatibility between CE and PE 2023-10-24 12:30:17 +03:00
Andrii Shvaika
0571f7e6dd Fix compatibility between CE and PE 2023-10-24 12:24:01 +03:00
ViacheslavKlimov
60900a0b96 Merge branch 'hotfix/3.6' into re-consumer-service-refactoring
# Conflicts:
#	application/src/main/java/org/thingsboard/server/service/queue/DefaultTbRuleEngineConsumerService.java
2023-10-23 16:41:38 +03:00
Andrii Landiak
f01cbe0e36 Refactoring ToCoreNotificationMsg: implement replacement for raw bytes messages 2023-10-23 10:10:02 +03:00
ViacheslavKlimov
cf951ee2b2 Multiple improvements for Rule Engine consumer services 2023-10-19 11:09:34 +03:00
Sergey Matvienko
92a2a90a0b ComponentLifecycleMsgProto implemented as replacement of raw byte encoding 2023-10-18 18:06:40 +02:00