Building

Started building at 2023/09/01 09:30:56
Using pegged server, 1441 build
Calculating base
Updating mirror
Basing run on 7.6.0-1441 8cce940e7c
Updating tree for run 01.09.2023-09.30
query is at 5d19d9b, changes since last good build: 
 5d19d9b23 MB-56068 Revise CPU setting
 eb6f386e4 MB-56068 Delay start-up until initial settings...
 9818b19cd MB-58453 Push phantom key skipping down to ...
 4c0b797d7 MB-58435 Quota accounting improvements
 68e4a3ab3 MB-57781 Modify fix
 e7e9abded MB-58433 Record new keys before insert
 07f8fac2f MB-58417 Correct logged messages
 8efa8a9aa MB-57758 Fix build break
 681d32794 MB-57778 Set num CPUs only on start
 7ae697cd0 MB-57717 Drop N1QL_SYSTEM_BUCKET at end of migration
 5c21302f3 MB-58317 Revise fix
 9bf203e7c MB-58387: Panic and no server response when request sent with invalid txid
 1d9ed7172 MB-58385 Add check for empty v-bucket map
 a7c86fd60 MB-51961 Simplify determination of cached result
 c7804ddc4 MB-58356 Synchronise access to scopes list
 0cdf062ae Fix unit tests
 dd9e9f48f MB-58322 Update unit test
 604be573b MB-58243: Upgrade to gocbcore@latest for fix(es)
 131518984 MB-58322 Report E_SEQUENCE_NOT_FOUND for KEY_ENOENT
 1011adf60 MB-58157 Make sure cached result values have appropriate "parent" values
 4e7068abd MB-58317 Improve bucket updater logging
 ef910c40d MB-22424 Add sequences to scopes
 5e2d70ffa MB-58301 Minor revision to fix
 ea1068caf MB-58301 Allow interrupted migration to resume...
 4e32d57bf MB-57394: Move to bleve@trinity-couchbase
 9f04ae667 Unit test update for China timezone (UTC+8)
 810c3103f MB-58292 Fix panic in writeControls
 273a03330 MB-39783 Add role semantics checking
 5de17a833 MB-58267 Defensive value exchange locking
 b472667ff MB-58268 Fix panic
 0df66cdb7 MB-58269 Fix message
 87cb669de MB-58078: Fix UDF unit test
 7b9c3bd9b MB-53384 Use result cache for covered index scan on primary index only when there is no spans
 b27db89e9 MB-57034 Remove go-curl reference
 2ec4523ca MB-58208 Add support for IF [NOT] EXISTS before object name
 e9aa4fee7 MB-57034: go.mod changes since go-curl is no longer a dependency
 7f10db8d6 MB-57034: Replace the use of go-curl in CURL() implementation
 7c96e2a16 MB-58150 Further limit cases we use OR clause filters for sarging index keys
 00a31ca17 MB-58192 Revise fix
 923eb522d MB-58078: Add unit tests for external JS UDFs
 0adbeb252 MB-11512 allow optional subclauses only if RECURSIVE keyword specified
 38087cd76 MB-58192 Fix syntax help data generation
 9153c8159 Revert "MB-58106 Further limit cases we use OR clause filters for sarging index keys"
 81b4cf886 MB-58190 Fix panic in cbq vim-mode.
 fbec2d6a2 MB-58106 Further limit cases we use OR clause filters for sarging index keys
 0d94a794a Revise syntax help generation
 4c5110f66 Further build script revision.
 d7ef13ff9 Fix build scripts
 6897f171e MB-58078: Add unit tests for N1QL managed JS UDFs
 18679cedc MB-56363 Provide line & column information...
 a1c69cd0b MB-57717 Initial changes for CBO stats migration
 fed5d08a3 Revise cbq build.sh
 a18188037 MB-53384 Fix unit test
 4a85c6319 MB-56363 Unit test update
 af002d685 MB-11512 Hierarchical / recursive queries using WITH
 cfe8f0c41 MB-57717 Create primary index on system collection (_query) when needed
 40f227425 MB-56363 Improve syntax errors context information
 15d740902 MB-58039: Add unit tests for Inline UDFs
 f67b8a40b MB-57717 skeleton changes to move scope UDFs to _system scope
 21468b1f7 MB-53384 Support of correlated subquery use primary index or sequential scan
 ffb8f3690 MB-58010. instead of query-cbauth use n1ql-cbauth
 9b4fed943 build repo update for tools-common
 04d61fa4d MB-51961 Do not use filter in case of cached Fetch/IndexScan3
 2f8e9615c MB-58004 Correctly report statement parsing errors
 f895e3c02 MB-57920: Add “correlated” field for subqueries in EXPLAIN and profiling output
 d36a39fb5 MB-58002 Ensure deadline is reset when not in use
 a8411ed9d MB-57167: [EXPLAIN FUNCTION] Complete query support for EXPLAIN FUNCTION on JS UDFs
 23c63078f MB-16789 Minor revision
 dbd0a89cf MB-16789 Add (basic) SQL++ syntax help to cbq shell
 02072ce6c MB-57964 Add limit on completed_requests plan size
 6572f31f0 MB-57961 Add multi-byte aware string functions.
 4cc51ada3 MB-57899 make sure that any panic releasing the execution layer does not affect completing the request
 862d99f76 MB-56524, MB-57489: [EXPLAIN FUNCTION] Modify EXPLAIN FUNCTION for inline UDFs
 2f6bf7775 MB-57240 Do not ignore error from formalization
 f5869782f MB-57904 Proper handle of WITH correlation in encoded_plan
 78740665e MB-57585: go mod tidy
 c43dc7774 MB-57883 Pick up the v1.0.0 releases of 'tools-common' sub-modules
 14593a484 MB-57904 Proper propagation of WITH clause correlation
 d3eb6e79c MB-57861 Error limits should only apply to DML errors
 b80a207b7 MB-57900 Proper check for max limit of number of spans when composing multiple spans
 2fd0e5841 MB-57869 Fix grammar.
 a39b25ece MB-57863 Proper formalization of query containing only an ExpressionTerm
 bacf2f90a MB-56553 Add max recursion depth for builtin recursivecte function
 bae69212c Uprade to bleve@v2.3.9
 8b809baa2 MB-57770 Protect against empty random document key
 ef41fac0e Revert "MB-57768 Protect against empty random document key"
 99c8e637a MB-57768 Protect against empty random document key
 25baa9f64 Use same plan flags for ORDER
 4b6fa6aba MB-28587 Better compare scan hierarchies when planning
 bc711c249 MB-26683 Provide function to return formalised statement text
 b0d8e974c MB-57766 : ALTER INDEX shows incorrect error message when index does not exist
 5fc31d0bd MB-57781 Proper save/restore this.node on builder
 e4eabf314 MB-53981 Correct case insensitive identifier handling
 592fd9d1a MB-57759 Correct optional argument detection.
 2df4a2072 MB-56816 Refine fix
 aa4abfb08 MB-56816 Revise fix
 9d4994982 MB-57729 Do not perform string replacement on the entire array key
 9ce01e124 MB-57716 Add DEFAULT as keyword
 286926e6d MB-57167 : [ EXPLAIN FUNCTION ] Add a BasicContext interface to expose to js-evaluator
 1be1ed7eb MB-56816 Revise fix
 1c9daa0c0 MB-56816 Serialise pool refresh & keyspace addition
 1e9b66da4 MB-57697 Add sequential scan completed qualifier
 f92d494b9 MB-56040 Correct memory rate FFDC
 f8f739443 MB-57666 With CBO do not always pick index scan with order
 778392163 MB-57647 Modify fix
 d249562aa MB-57647 Do not use covering index scan for system indexes
 50b79e362 MB-57645. Pick latest gocbcore
 2d9ea3eb9 MB-57646 Fix unit tests
 1021eaa53 MB-21959 Implement switchable duration output format
 0ded91abd MB-57473: Allow query to read from replica only if cluster supports it
 44d77b8e3 MB-39484: Remove N1QL support for password-less buckets.
 1c7ad86b9 MB-39494: Remove N1QL support for password-less buckets.
 15a1ff8c6 MB-57598 Revise test cases
 94badcd01 MB-57598 Handle additional date component separators
 ec5677965  MB-56286: support indexer agnostic securityChange handler
 4928ec67d MB-57284: Upgrade to gocbcore@v10.2.5
 727215138 MB-57348 defensively check that dequeued request is populated
 883c16a1c MB-57394: go mod tidy
 8a949f149 MB-56348 defensively check that dequeued request is populated
 61a3f1c38 MB-57346 Handle OR clause with IS MISSING better in index advisor
 4f9d74e58 MB-57197 Fix unit test failure
 c0cb02767 MB-57197 Enhance date functions.
 95bbd2ca0 MB-57287 Revise implementation.
 a9f727a16 MB-57240 Handle Subqueries separately
 d3ab2c7d8 MB-56916 Use arrays to insert redaction tags...
 473727c38 MB-57287 Support XML output format
 59b270203 Update build.sh
 e02d1dffe MB-57240 Proper detect and mark correlated subqueries from encoded plan
 a3d3c2428 MB-56912 : Add -advise option to CBQ shell
 ab17850ed MB-57197 Revise date functions.
 f9fc60614 MB-57229 Revise perms for admin endpoints
 b58e29e35 MB-57209 Add mask and fixedlength filter options
 1fa968ca1 MB-57209 Redact function.
 00789c76f MB-57187 Follow-up fix
 cde105e6b MB-57187 Additional updates for apple M1 chip
 b9873f4fd MB-57187 Update unit test for apple M1 chip
 34cb2e313 go mod tidy
 2ef5758b0 MB-57169 Retry host to node ID resolution...
 1d42b0cdb MB-45267 Revise fix
 b070b735f MB-57153 Revert to path elements in E_DATASTORE_INSUFFICIENT_CREDENTIALS
 4d6e8b4e8 MB-45267 Limit use of errors.NewError
 b3bb70998 MB-57146 Fix admin endpoint redaction
 109987083 MB-54782 Revise fix.
 1f386d73e MB-56524: Support for EXPLAIN FUNCTION statement.
 58018b6e3 MB-57116 Change threshold qualifier to check ...
 095528bb8 MB-56128 Provide pretty option for admin endpoints
 c8be2f3af MB-57052 improve fix
 ec5babb8b MB-56553 Rcte builtin function
 aa53dcd52 MB-57102 Use a separate structure for WITH clause
 6dac1bddb MB-57052 check bucket cleanup status every minute
 2654b7491 MB-57038 Address memory usage for host to UUID mapping
 e503a6faf MB-56956 Don't close operators more than once
 bb01d5d1c MB-57103 fix error codes
 9564ee87d MB-56968 Revise new key checking limits
 f8afe6b76 MB-57100 Add node_uuid function
 aed72242c MB-57099. Report actual memory Usage percent with respect to node quota
 6fc6dafba MB-57072 Ensure state isn't released whilst in use
 a9ea549d0 MB-57073: [query] Fix missing mutationCount in DMLs on system keyspaces
 6f871121e MB-56040 Correct type for memory usage
 e9994553c MB-57048 Report host resolution failure in PrepareTopologyChange
 c3648cace MB-57046 Add FFDC count to vitals
 2e2c8e832 MB-56997: FIx hang when a query plan is executed via ExecutePrepared() and there is a fatal error in the root operator.
 11b6d4596 MB-57022 Add admin endpoint to access diagnostic logs
 4c3c3756f MB-56771: fix errors in regulator stats
 c11b3c26d MB-56372: Fix mistaken assignment in ExecutePrepared()
 cfe0da5e3 MB-56816 Enasure bucket updater is started when reloading the namespace.
 637f99c1f MB-56771: update metrics_metadata.json w/ regulator metrics
 6180eaf9d MB-56933 emit 'n1ql_' prefixed regulator stats
 a60854d37 MB-56988 Handle FALSE WHERE clause better
gometa is at 96e745a, changes since last good build: none
ns_server is at b514dd1, changes since last good build: 
 b514dd198 MB-58347: Verify warmup state of memcached when getting status
 c1089421f MB-57062: Add cm_rebalance_stuck metric matching the alert
 edc35b002 Make memcached_connections alert resilient to lack of stats
 e2f8aedb6 MB-58451: Handle service failover correctly.
 d8b7f9bcd MB-58278: Redirect noisy progress reports.
 3657bcf32 MB-55822: Add CBO tracing log file analytics_opt.log to cbcollect_info. Also, analytics_shutdown.log is being removed from cbcollect_info since it has been removed since 6.5.0.
 d4ca3ad35 MB-58303 remove pre 7.1 clause from ns_janitor
 dab77bbd7 MB-58303 remove pre 7.1 message from ns_config_rep
 f2fb4ce41 MB-58303 remove pre 7.1 code from ns_server_cert
 69121066c MB-58303 fix comment in index_settings_manager
 ef4874862 MB-58303 remove unused ns_server_cert:set_cluster_ca
 1442a011e MB-58303 unconfuse comments in menelaus_alert
 fa5cccd89 MB-58303 remove pre-7.1 actions from auto_failover
 af97396eb MB-58303 remove hot_keys_keeper
 9f2247e46 MB-58303 remove more pre 7.1 code from ns_cluster
 60dd329af MB-58303 remove sanitization of sasl_password
 3bcab2955 MB-58303 remove maybe_register_on_epmd from cb_dist
 a7f04a6ce MB-58303 remove handling of undefined chronicleInfo from
 dc98756b1 MB-58303 stop sending clusterCA with engageCluster
 3f423147f MB-58303 use ns_config compat version for query settings
 891077931 MB-58303 stop sending {get_keys, _, _} call to ns_memcached
 9a5098080 MB-57931: add queryNodeQuota to cluster-init
 e28789782 MB-57182: put config_profile into cbcollect
 f3bb1e17d MB-58303 simplify ns_janitor:maybe_pull_config
 e31a581b5 MB-58303 drop pre 7.1 roles
 a2d9048f0 MB-58303 remove cluster_compat_mode:is_version_71
 ac12fb611 fix intermittent breakage of ns_rebalance_observer_test
 babeb6a4d Revert "MB-57758 pass --num-cpus  to query service"
 c3d340ad4 Remove pitr keys from community upgrade path
 75def4423 MB-57955: Add new bucket keys to upgrade path
 7c999ed1c MB-57955: Add versionPruningWindowHrs bucket parameter
 844b1a384 MB-57955: Make max_cas available via bucket get api
 277ed26b7 MB-57955: Add per vbucket max_cas recording
 de3f1a50a MB-57955: Add enableCrossClusterVersioning param
 2e6792cd1 MB-58339: apply cluster.admin.security.external permissions
 78f64b620 MB-55578: Expand health monitor behaviour test to include index monitor
 d13a8a099 MB-55578: Expand health monitor behaviour test to include kv monitors
 8716fa289 MB-55578: Remove kv_stats_monitor get_buckets()
 c516dce18 MB-55578: Add meck new/unload convenience macros to allow meck re-use
 9faa172bb MB-55578: Add health_monitor behaviour cover test
 f5924dd4b MB-55578: Add health monitor integration test
 2aa1470fe fix typo in chronicle_upgrade
 6e5d80255 MB-58303 remove cluster_compat_mode:is_cluster_71
 962ddb81a MB-58303 remove code dealing with cert_and_pkey ns_config key
 9d1f9952e MB-58303 [cluster_tests] remove pre 7.1 code
 2cf76f6a2 MB-58303 remove pre 7.1 code from menelaus_web_certs
 69a2c824a MB-58303 remove group auto failover code
 eade6ce8e Remove examples from default_profile
 7d9734aea MB-58303 remove menelaus_util:assert_is_71
 42bee98fe MB-58303 remove pre 7.1 code from compaction modules
 425d8834f MB-58303 remove support for non trimmed dcp connection long names
 8b12613bf MB-58303 remove unused define VERSION_65
 e3eade32d MB-58303 remove pre 7.1 logic tests from auto_failover_logic
 9c3df5d19 MB-58354: [SAML] Audit saml settings changes
 231a58499 MB-58303 stop consulting ns_config when getting cluster compat version
 34245b0f6 MB-55578: Add type spec for health_monitor:(local|node)_monitors()
 eaefa1d3c MB-55578: Enhance type spec for health_monitor behaviour state map
 8821a735c MB-55578: Allow index_monitor to scale refresh interval below 1s
 d29501a2c MB-55578: Make index_monitor implement health_monitor behaviour
 38d5dae2c MB-58303 remove ns_config:get from chronicle_upgrade
 cb4323de0 MB-58303 make all services prehistoric
 a698bf1ad MB-58303 remove ns_config upgrades up to 7.1
 424639900 MB-58303 remove unnecessary ns_config:get from memcached_permissions
 b783122df MB-58303 remove pre 7.1 code from capi_crud and
 143948a4d MB-58303 remove now unused is_cluster_70, is_version_70
 5a8c2eddf [cluster_tests]: Increase timeout for loadingSampleBucket task ...
 57ae99f07 MB-58303 get rid of the pre 7.0 code in tombstone purger
 7a7bccb05 MB-58303 remove dead config sync related code from ns_janitor
 1ab935b58 Avoid chronicle:get/2 from within a chronicle transaction.
 bd5ccede8 fix broken 7.2 -> Trinity config upgrade
 62ec931b2 MB-58303 get rid of ns_config related code in ns_bucket
 2421c0604 MB-58303 get rid of redundant chronicle_compat:set
 9be509249 MB-58303 disallow passing ns_config into chronicle_compat:get and
 e56fd5b65 MB-58303 remove some dead code from chronicle_compat
 36039ed58 [cluster_tests] Move orchestrator utilities to testlib.Cluster
 2615f487b MB-57366: Add kv resident ratio growth reaction
 0fa785660 Add number validator type and num settings2 type_spec
 f1ef61b63 MB-57033: Resource management configuration
 ad7199253 MB-57033: Add guardrails modules
 3084b675e MB-56860:  XDCR visual adjustments
 1e00710bc MB-56513: use optional syntax to ignore error
 cb0ebe3aa MB-56513: set default spVerifyRecipient and spAssertionDupeCheck
 3f877d900 MB-58303 clean some dead code from chronicle_master
 d50d08b66 MB-58303 remove no longer used we_were_shunned message handler
 36593ab11 MB-58303 remove chronicle_compat:enabled
 dbe79eb16 MB-58303 remove compaction_daemon:chronicle_upgrade_to_71
 e0c462182 MB-58303 remove ns_bucket:chronicle_upgrade_to_71
 99eb9c911 MB-58303 remove ns_ssl_services_setup:chronicle_upgrade_to_71
 4a03753f1 MB-58303 remove after upgrade cleanup routine
 06a52b79e MB-58303 get rid of upgrade to chronicle
 e8b2e7a76 MB-57758 pass --num-cpus  to query service
 955a8dd3d MB-11445: Add alert for memcached connections
 0edf2d00a MB-58303 get rid of unused stuff in chronicle_compat_events
 1c5fc6326 MB-58303 get rid of unused ns_cluster:prep_chronicle and
 1c562dad5 MB-58303 get rid of chronicle_master:upgrade_cluster
 f5520179a MB-58303 remove bits of pre 7.0 code here and there
 042b64f13 MB-58303 get rid of menelaus_stats_gatherer:invoke_archiver/3
 1da10ef8f MB-58303 remove 6.6 hack from netconfig_updater
 0362f4af2 MB-58303 remove ns_rebalancer:push_ns_config which is obsolete
 85610856f MB-58303 remove some 7.0 code from ns_cluster
 0d5e36cc4 MB-58303 remove unused mock
 83ba89cf7 MB-58303 remove upgrade hack from menelaus_web_alerts_srv
 9a6cc5b1b MB-58303 remove collections:enabled
 8da7251f6 Revert "MB-52267 Add server group info to topology aware messages"
 245c7a270 MB-57062: Add kv stuck rebalance alerts
 0f50eb865 MB-24487: [SAML] Add groups and roles to...
 b603e6fc6 MB-58213: dispaly samlErrorMsgId
 17de88ad4  MB-24487: [SAML] Tag logs that can contain user data...
 949baba3a MB-24487: [SAML] Audit login failures when...
 858b9f0d5 MB-24487: [SAML] Redirect user back to UI if authz fails
 80e97a12d MB-24487: [SAML] Refactor saml /consume handler
 e58ceeeaa MB-48217, MB-51082: [SM] Rename "state" in...
 062c4abe7 MB-58303: Add TODO comments in ns_online_config_upgrader.
 4c8261416 Merge remote-tracking branch 'couchbase/neo'
 38974e73c Merge remote-tracking branch 'couchbase/7.1.x'
 2d38ec170 MB-58069 Report raw cgroup cpu stats
 000824e2e MB-58335 [BP]: Upload cluster config on memcached restart
 9588ebe9e MB-58303: Remove menelaus_users unused upgrade paths.
 378157020 MB-57435: Remove menelaus_users upgrades from cluster_compat_mode.
 686c5d783 [cluster_tests] Set requests timeout to 60 by default
 76c7c872d MB-48217: [SM] Don't do sync stop in cb_gosecrets_runner
 cde1b011b MB-48217: [SM] Add more unit tests
 6253c7570 MB-48217: [SM] Make sure password can be reset to ""...
 96713fc24 MB-48217: [SM] Support for password change in case when...
 24fc5bb27 MB-58303 n1ql capabilities are no longer in preview.
 60e071c46 MB-57435: Fix offline upgrades to use list_to_tuple(min).
 e6e925e96 MB-57222: remove SWEET32 vulnerable ciphers
 7bbe10d8f [cluster_tests] Fix short_exception formatting
 8dda4341a MB-58127: [cluster_tests] Add __repr__ implementation for...
 e6abd131c MB-58242: Return log message with error when failing to get DCP stats
 9bcc2f379 MB-58241: Log node on which we cannot retrieve DCP stats
 6dcff301c MB-58303: Remove uses of ?VERSION_65
 fc940064d MB-58303: Purge references to VERSION_66.
 4282d9734 MB-58303: Remove 6.6 from cluster initialization path.
 f54cc91f0 MB-58303: Remove query_settings_manager upgrade to 70 path.
 2a3172f9d MB-58303: Remove index_settings_manager upgrade to 7.0, 7.1 paths
 67b2e6c7e MB-58270: Specify settings to those of min for X_settings_manager
 f29978606 MB-57435: Prevent offline upgrades from 7.1.
 8dc926771 MB-58213: [SAML] Fix a typo in UI
 55a18a3a0 MB-58230: Fix unicode reload_node_cert audit
 9739f5a41 MB-58213: add ability to disable spAssertionDupeCheck
 5b6d040e5 [cluster_tests] Merge requirements in order to ...
 972a05e29 [cluster_tests] Cosmetics
 32bba22bf [cluster_tests] Add --verbose arg (-v)
 a96d30583 [cluster_tests] Refactor safe_test_function_call
 6f195beb1 MB-57854 Add current rebalance report to cbcollect_info
 32c877366 MB-57854 Add a function to collect current rebalance report
 dba6d397b MB-55578: Make disk failure auto failover configurable down to 1s
 9beec7778 MB-55578: Make kv_stats_monitor implement the health_monitor behaviour
 09a3b61b6 [dialyzer] Use proplist type in place of pattern
 d2b42284f MB-58091 _default collection should be "metered"
 58cd24da2 MB-58091 "metered" should not be used on-prem
 e279eeda7 MB-57260: Use bucket priority for failover
 ae61e4dfd MB-57260: Use bucket priority for rebalance
 4ce9bacac MB-57260: Introduce bucket priority setting
 121ebd6c6 MB-57435: Use MIN_SUPPORTED_VERSION for config upgrades
 61adc4c3e MB-57435: Remove ns_bucket:upgrade_to_66.
 9fca7c350 MB-57435: Remove menelaus_web_alerts_srv:upgrade_to_70/71.
 cf149c921 MB-57435: Bump min_supported_version to 7.1.
 0558b1e9c MB-58213: handle samlErrorStatus and samlUserName params
 c52f0fe67 MB-58045: Remove support for TLSv1/TLSv1.1.
 930f1948f MB-58176 - change screen text to indicate phone home mandatory in CE
 38fe4c18f MB-58228: Ensure the 'config' dir exists.
 7de1bffc9 MB-57867: Fix erlc build flag: bin_opt_info
 17740d763 MB-57843: Return error when domain is invalid in ...
 3aa1d687e MB-55508: Add ability to do apply config on buckets in parallel
 685d8d8be MB-55508: Do single config sync when janitoring a set of buckets
 29dac0c21 MB-55508: Janitor a set of buckets at a time during failover
 6a68fa96b MB-55508: Add ability to run query vbuckets in parallel
 fe3385854 MB-55508: Introduce cleanup_buckets() in ns_janitor
 28c1d9132 MB-55508: Split config and memcached steps in janitor cleanup
 5f0e293a4 MB-55578: Pass BaseMonitorState into child monitor init
 671a6bf76 [cluster_tests]: Wait for node to be ejected ..
 5ebfe79d9 MB-55578: Store MonitorState correctly in health_monitor:handle_call
 4d7ecdfec MB-55578: Simplify kv_stats_monitor refresh mechanism
 7145e80dc MB-58168: Only get statuses from ns_memcached once in ns_heart
 9604df586 MB-55578: Only get statuses from ns_memcached once in kv_monitor
 0408af843 Revert "Check for duplicate REST keys"
 d3ff245d6 MB-58218: Correct typo causing bad match in kv_stats_monitor
 52e100754 Check for duplicate REST keys
 c107fefe9 MB-57993: Reduce default num_vbuckets to 16 ...
 52f7a613a [cluster_tests]: Remove setting num_vbuckets.
 aa3ee06dc MB-56332: Remove throttle support from internalSettings
 be6b7b7f0 MB-57924 Allow resetting collection maxTTL via set_manifest
 999b2fd49 MB-56513: use consistent radio boxes
 7f4331a76 MB-58172: Make "sendStats" mandatory for community edition
 3c93814a6 MB-57994 Check for duplicate params in request
 ccbde3ab7 [cluster_tests] New clusters still need to be checked...
 ac5aaae0b [cluster_tests] Add MasterPasswordState cluster requirement
 74e1f68bb [cluster_tests] Get rid of annoying print from xmlschema
 7491c5737 [cluster_tests]: Test travel-sample loading on ...
 9a2a81a05 MB-55578: Make kv_stats_monitor use map rather than state
 14bd6194e [cluster_tests] Fix group_testsets
 ed0bcceee MB-24487: [SAML] Minor UI corrections
 b381b8e26 MB-57501: Test "cert will expire soon" alerts
 03db2921a MB-57493: Add tests for EC key, RSA key, and PKCS12 certs
 8cfa4a3cf MB-24487: Redirect user from uilogout to /saml/deauth
 69788bc5e MB-52181: Fix cert expiration alerts
 73ea2c999 MB-48217: [SM] Add cluster tests for password via command
 770a13db4 MB-48217: [SM] Fix API call that comes right after another failed...
 a4d91613e MB-48217: [SM] Add unit-tests for upgrade from 7.2
 c35d6cfbf MB-48217: [SM] Add secret management settings API
 7a82dc838 MB-48217: [SM] Manage datakey using external cmd
 1cbf5eed2 MB-48217: [SM] Support for reading master password via cmd
 cfa46b529 MB-48217: [SM] Add support for gosecrets config reload
 f6861d30d MB-48217: [SM] Add some gosecrets unit-tests
 a3ba91b12 MB-48217: [SM] Split encryption_service code into
 fd29ad341 MB-48217: [SM] Add some tests for secret management
 14cc0eac4 MB-48217: [SM] Log unexpected messages in encr service
 2b8c184ba MB-48217: [SM] Send panic reason to babysitter
 feb076bfd MB-48217: [SM] Add logging in gosecrets
 8a0575ddd MB-48217: [SM] encryption_service refactoring
 55d431b4c MB-56513: single sign-out UI
 2b92dffd4 [cluster_tests]: Fix incorrect f-string.
 2f223cb34 [cluster_tests]: Rename function.
 562469c10 MB-55578: Always store refresh_interval in health_monitor state
 3457e2b05 MB-55578: Reduce minimum auto-failover timeout value to 1 second
 beb777865 MB-58148, MB-57260: Fix removing per-node keys on node ejection.
 f0d9e53a7 MB-57924 Added unit tests for collection maxTTL
 bdf5283d3 MB-57924 Provide way to reset collection maxTTL
 b3101102a MB-57924 Pass maxTTL=0 to memcached to disable TTL
 fb23a827e Update the release for newly added stats
 3355e3dfc MB-58138 Add replacment for sysproc_cpu_utilization
 9fce76b4b Revert "MB-55578: Make kv_stats_monitor use map rather than state"
 7655963c9 Revert "MB-55578: Simplify kv_stats_monitor refresh mechanism"
 048fe8e38 MB-55578: Simplify kv_stats_monitor refresh mechanism
 08d14b47e MB-55578: Make kv_stats_monitor use map rather than state
 e50da99a1 MB-55578: Correct health_monitor behaviour callback specs
 e2fb54ed5 MB-55578: Simplify refresh logic in index_monitor
 1c49131b4 MB-55578: Pass health_monitor State to can_refresh
 56e00a380 MB-55578: Use map in index_monitor state rather than record
 47aed640f MB-55578: Move specific monitor testing earlier in test
 f4ed5693f MB-55578: Scale monitor refresh with auto-failover timeout < 5s
 fd1bb33c0 MB-55578: Cache monitors in node_status_analyzer state
 b30357f63 MB-57989: Use nodes_wanted in ns_heart:status_all rather than nodes()
 6728a4d17 MB-58009 fix incorrectly written label_to_service
 105b1f886 MB-58009 cache size overrides should work for any services, not
 fbedde260 MB-58036 Add query sequences roles.
 343362ea6 MB-56860: add retry logic and show result in textarea
 98897e72d MB-56860: show up all connectionPreCheck errors
 ee82d8d39 MB-58091 Bump manifest UID on upgrade
 b3fd8118a Filter out deleted files from git diff used by erlang formatter
 6c346f4f7 MB-55587: Move child monitor test setup after parent monitor setup
 92c062e06 MB-58087: Log auto-failover logic state when we create it
 91e0f2fef MB-58087: Use correct timeout when updating auto failover logic state
 37434058c MB-57773 Adding secrets management status to cbcollect_info
 4409cf85b MB-57966: Persist query completed-max-plan-size settings
 00ed8dd4a MB-55578: Cache local monitors in node_monitor state
 d28ea1e0b Fix typo: apply_on_belalf_of_identity -> apply_on_behalf_of_identity
 fba7f2f82 MB-57499: Add tests for password hash settings
 11e168e38 MB-57572: Check status of write of service map on failover
 a767c8f5a [cluster_tests]: Test delta-recovery mid storage_mode migration.
 e2d5b7ba8 MB-56513: delete spVerifyRecipientValue if Recipient is custom
 17d9e006a MB-56513: disable the form fields correctly
 c2ae55f49 MB-56513: reset form errors on submit
 b49541d5e MB-56513: hide appropriate fields when File or Custom Node is selected
 cb98d80ba MB-56513: use mn-file-reader for the Trusted Fingerprints
 b6f3916e5 MB-56513: add http_one_time parameter support
 f03c1d17a MB-56513: use mn-file-reader for other file fields
 a837f9e6a MB-56513: upgrade mn-file-reader and use for Load IDP
 5f7a8fe7f MB-56513: allow admin to create external user if saml is enabled
 5c5512810 Revert "MB-56073: Revert "MB-55588/MB-55747: Fix couch_docs_fragmentation for old endpoint""
 27cb6e322 MB-57572: Update service map after bucket prep.
 5f167a73a MB-24487: Logout saml users when saml gets disabled
 ef47d7741 MB-24487: Fix cert validation
 0fdc4992f Put licence info in order for tests
 ec7dc5e66 [cluster_tests] Disable colors when not in terminal
 828fdcfb7 MB-57522: Add cluster tests for int pass rotation
 8c8c6220c [cluster_tests] Move data_path to node
 d09f4c822 MB-56513: asterisks are fine for spKey
 4d59d72bd MB-56513: make form fully reactive and update on save
 9a46ca294 [cluster_tests] Add "services" cluster requirement
 436283945 [cluster_connect] Add --start-index support
 692047639 [cluster_tests] Add some improvements in poll_for_condition
 9aa565ab9 Revert "[cluster_tests]: Test delta-recovery mid storage_mode migration."
 2aea08da4 [cluster_tests]: Test delta-recovery mid storage_mode migration.
 bc15604ee MB-55579: Disable storage mode migration in serverless.
 4903ec44c MB-55579: Bucket storage mode migration.
 a67a42b54 Add bucket_test in cluster_tests runs by default
 1eae3369d MB-57800 Add bucket capabilities for system collections
 332d39fd7 Merge "Merge remote-tracking branch 'couchbase/neo'"
 2b39dc1f7 Return "numVBuckets" instead of "numVbuckets" in  ...
 76e7f5837 Merge remote-tracking branch 'couchbase/neo'
 b74a2d0fd MB-57945: Update metrics metadata
 1e4510898 [cluster_tests] Add import time to prom_sd_config_test
 6c7409707 MB-58003 Add unit test for _system scope upgrade.
 5aff19b25 MB-46500: [users_backup] Make include and exclude mutually...
 8aef25d27 MB-57800/MB-57683: Upgrade on-prem buckets to use _system scope
 4aee4425d MB-57800/MB-57683 Add _system scope to on-prem configurations
 624d1d9b3 Erlang formatter changes for commit hook
 9d94f6ac9 MB-55578: Make specialised health monitor handle_cast/info return state
 ee3f17a88 improve the printout of build_dynamic_bucket_info_test
 97df95248 MB-55578: Add extra unit tests to dcp_traffic_monitor
 c7239c8fa MB-55578: Make auto_failover_tick_period configurable
 367c37304 MB-55578: Move auto_failover tick_period to state
 45209fbc3 MB-55578: Set inactive time to 2x refresh_interval
 d65a649c9 MB-56860: Add PreCheck to adding a remote cluster
 84e2d0ccb MB-57332: Add XDCR Filter Binary Checkbox
 777cc8129 MB-56513: Import mnUserRolesService
 4da7e339e MB-55578: Make health_monitor refresh intervals configurable
 ed432de08 MB-55578: Pass child monitor state to health_monitor:send_heartbeat
 7eb85244b MB-55578: Calculate/store refresh_interval in health_monitor
 65c52bff0 MB-53073 Add stats to CbAuth cache
 3bfa7a216 MB-57887 No expiration for _system scope collections
 88858443c Verify alt addresses available
 8ed917664 MB-56513: Integrating SAML SSO configuration UI
 a317f7acc MB-31806: Allow modification of collection maxTTL
 cf902564a MB-57886: Parse pressure statistics from ns_server.stats.log.
 b1bedff27 MB-57826: Add provisioned config profile
 e289973c7 Reformat collections.erl tests
 71d9e4a3b MB-57589 Change VERSION_ELIXIR to VERSION_TRINITY
 063702a0a Fix formatting for collections:verify_oper
 3012971bf Add comment explaining how dcp_proxy:nuke_connection works
 2e817f7dc Fix erlang diff formatter for file creation
 dd4c2fbb5 MB-56470 Unblock ns_audit in case memcached is slow
 216235965 MB-56470 Continue sending logs if einval error happens
 11134ee4e MB-53072 Make cbauth cache size configurable
 7ac670caf MB-57790 Use min replicas when loading sample buckets
 4eb03ac55 Merge "MB-57614 Merge remote-tracking branch 'couchbase/7.1.x'" into neo
 1dca0de1c Merge remote-tracking branch 'couchbase/neo'
 0158e2e43 MB-57614 Merge remote-tracking branch 'couchbase/7.1.x'
 7948e4efc MB-57682: Verify couch_config:set succeeded.
 95d31a224 Make erlang formatter check mode run git diff quietly if run under hook
 5396c86d2 [cluster_tests] Ability to specify seed for tests
 81495f57c MB-55578: Rename peers_changed fun in health_monitor to config_changed
 52fa67e06 MB-55578: Remove health_monitor:common_init
 337d9e21c MB-55578: Store refresh_interval in health_monitor state
 def3fbd5c MB-55578: Pass MonitorState to health monitor handle_call impl
 9eaa5ca1d MB-55578: Pass MonitorState to health monitor handle_info/cast impl
 f8acab984 MB-55578: Add can_refresh function to health monitors for test/assertion
 a3195c2ca MB-55578: Add basic unit test for health monitor functionality
 b3ebb5db0 Fix incorrect timeout value in wait_for_rebalance.
 0849ceea8 MB-55578: Add and use health monitor behaviour
 26e37da45 Code coverage for unit tests
 b1f5dd070 MB-57674: Ensure cluster_tests doesn't crash on test setup fail
 d81ac6f37 MB-57677 Fix misc badmatch error
 17b294dee Merge remote-tracking branch 'gerrit/neo'
 ff0b9860d MB-57596 Add sysproc_start_time to stats description file
 b5e870fb2 MB-57639: Check collection manifest has changed before updating
 e3392a122 MB-57632: Add metrics_metadata.json entry for sys_disk_time_seconds
 a7662afe1 MB-55948: Revert "MB-55912: Temporarily disable history for default collection"
 6ab9d368c MB-56030: Revert "MB-55960: Temporarily disable history for all collections"
 a45139e1a Merge "Merge remote-tracking branch 'couchbase/neo'"
 fce1b5128 MB-55376 allow clients to request periodic heartbeats on all
 62fa828a8 MB-55376 allow client to enable periodic heartbeats sent via REVRPC
 633ead703 MB-57591 Report stats to prometheus
 cd0d04312 MB-57472 Add "readFromReplica" clusterCapability
 1291fd7d7 [BP] MB-57614 Fix case_clause issue in stats
 ccffbbedb Merge remote-tracking branch 'couchbase/neo'
 07ad14836 MB-57449: path_config_data_dir default value for cbcollect_info
 67bbf88b4 MB-57454: Replace _deleted keys with Default in dump-guts
 d4e59f751 Add -o option to cluster_tests help
 5d9814412 MB-56842 Poll for desired condition
 261185c86 Merge remote-tracking branch 'gerrit/neo'
 bb3b155cb MB-57484 Merge remote-tracking branch 'gerrit/7.1.x'
 e32c7d78d MB-57484 [BP] attempt to delete unused buckets from memcached before
 729278257 MB-57470 Add subdoc.ReplicaRead to bucketCapabilities
 ea9fa785c MB-56023: Allow autofailover if only a single kv-node remains
 790a31f17 [cluster tests] Add some time stats
 15f87e792 [cluster tests] Turn on debug output in wait_node_up...
 c12e232dd cluster_tests: Skip subsequent tests after a test_teardown fail
 4733dd08f MB-57312: [ldap] Add formatting for {asn1_enum, N} errors
 74e207034 MB-57476: cluster_run_lib - Catch all exceptions in wait_node_up
 9899cc690 Merge remote-tracking branch 'couchbase/neo'
 82d43a3cd MB-57331: Merge remote-tracking branch 'couchbase/7.1.x' into neo
 cf57872cd MB-57331: Set data key after recovering master password
 25263ee69 MB-57331: Fix restart of gosecrets after password change
 0e5240f9a MB-55508: Add test condition for failover
 e7f25fd37 MB-55508: Bulk config updates over buckets for failover
 77eee2973 Merge "Merge remote-tracking branch 'couchbase/neo'"
 aa73fe16c [cluster tests] Intercept output from tests
 1eb9b00a3 [cluster_test] Print traceback after result
 e7df2f0f0 Merge remote-tracking branch 'couchbase/neo'
 fab2a286c MB-57006: Upgrade GOVERSION to SUPPORTED_NEWER
 b241b6767 cluster_tests: fix formatting and remove redundant code
 6d0c81cef MB-57337: Show vbucket under cursor in rebalance plotter
 903f3967a [cluster_test] Print traceback with variables
 20909f6a8 MB-46500: [Users backup] Return users and groups that...
 4c1718c5f MB-56825 Move minReplicaCount to new endpoint
 fe1f878a9 [cluster_run]: Shutdown nodes gracefully.
 481b4143f MB-57361: Fix broken auto-failover when a single monitor is in use
 e5bce357c [cluster tests] Flush the output of test name
 7972001b4 [cluster tests] Show test duration
 8e6ed2964 MB-56447 Don't reply back to the sync requesters if a refresh is in progress
 e6e643386 MB-55347: Add magma props only when it's a magma bucket.
 da20c2998 MB-56842: Add serviceless node test to cluster_test suite
 af9293eee [cluster tests] Pretty print list of testsets
 5fc651d34 Allow rebalance plot to show rebalance with unfinished moves
 ae25fffe4 MB-56839: Check raw_ram_quota for loading into existing bucket
 7b2963e1a MB-56447 Fix ns_server exit when memcached_refresh becomes irresponsive
 f29a1b46b Rename rebalance-flow.txt->rebalance-flow.md
 3c0c66ec6 Erlang diff formatter using emacs
 bad0cfdce MB-56833: assert if rebalance succeeded in Cluster.rebalance
 13800889c MB-56834: Keep track of connected nodes for REST requests
 dcbe4abc6 Fix error masking during bucket props update.
 b97959cee MB-57162: Accept message types from pre-elixir service_agents ...
 b1b97c701 MB-56034: Make a script to generate index workload.
 5500331dd Remove ns_dialyzer_obsessive from CMakeLists.txt
 234174cd2 MB-57065 Add cluster_test for /prometheus_sd_config
 d3b3a9de6 cluster_tests: Wait for rebalance when create/delete bucket
 9690824ba cluster_tests: List specific unmet requirements
 21f3e0652 MB-57258: Add analytics settings for blob storage
 e6cde4318 cluster_tests: Make connected_nodes requirement strict
 31ee0b591 Add basic test for on-behalf-of
 58902e0eb extract_effective_identity should take into account whole...
 40376c906 Merge "Merge remote-tracking branch 'gerrit/neo'"
 793ab259f Fix broken sigar unit test on mac
 4f30e5c24 Merge remote-tracking branch 'gerrit/neo'
 b2241b51c MB-56894 allow all full admins to access memcached on behalf of other
 5b2084a41 MB-56894 allow all full admins to act on behalf of other users
 45deb126f MB-56825 Add unit tests for minimum replicas
 82f54b0a2 MB-56825 Support min replicas for buckets
 01c36fd15 MB-57176: Couchdb incorrectly started serverless
 56a1bb6b7 Update dcp_traffic_monitor comment.
 a5c55c8dc Fix usage of needs_attention.
 fc0cca488 MB-46500: Add audit for users when restoring them from backup
 9a4c9b3a0 MB-24487: Make saml integration EE only
 e4ed85f6f MB-46500: Add tests for users backup/restore
 ef80c4661 MB-46500: Add canOverwrite param for rbac backup restore
 e70abc2d0 Merge remote-tracking branch 'couchbase/neo' into master
 60815afa2 Merge "MB-57006: Merge remote-tracking branch 'couchbase/7.1.x' into neo" into neo
 c7c4e8d47 MB-57006: Merge remote-tracking branch 'couchbase/7.1.x' into neo
 67ad7cf90 MB-55825: update FTS node capacity
 65db1e773 Merge "Merge remote-tracking branch 'gerrit/neo'"
 113228494 MB-56730: Flag a missing ns_server_monitor update from a node.
 bdeb67bc7 Merge remote-tracking branch 'gerrit/neo'
 a97369039 MB-55376 add NodeUUID to the cbauth payload for external clients
 b89f81bbc MB-55376 cbauth for external clients
 869907821 MB-56412 Provide reason for needing rebalance
 198fc88eb Add support for filtering ns_test via cmake
 772f5cdb9 MB-57111: Add mcstat bucket_details to cbcollect
 be8f617c9 MB-52912 Revoke n1ql.meta!read perm from bucket_admin
 6ae0a3874 MB-55576: Take into account services when selecting orchestrator node
 a4f4c69a8 Make sure ldap tests don't affect each other
 026f1d514 MB-56911: Add mb_master override to force orchestrator move
 3975166d6 MB-55576: Refactor mb_master priority to use common functions
 78d0fef01 MB-55576: Use common setup and teardown for mb_master tests
 a63709b2c MB-55576: Add mb_master state diagram
 d3d3e679f MB-55576: Add strongly lower priority test
 1395c86e3 MB-55576: Expand higher_priority_node test in mb_master.erl
 c5b36e7f2 MB-57087 "wrap" autogeneratedClientKey so it gets sanitized
 146dcfd9e Merge remote-tracking branch 'couchbase/neo'
 9d1f8c517 Change "units" to "unit" for sys_pressure_total_stall_time_usec
 f444729fa MB-55576: Format mb_master.erl
 9472bdbce MB-55576: Remove MB-33750 workaround
 8bd657dcb MB-56916 Use POST for gathering Query request data
 36692bf21 Add xmlsec1 dep in cluster_tests README
 7562089e8 MB-57063: Merge remote-tracking branch 'couchbase/7.2.0'
 29b25d716 MB-57063 "wrap" autogeneratedKey so it gets sanitized
 776a5204a MB-57068: Upgrade GOVERSION to SUPPORTED_NEWER
 0dca42ad5 Correctly remove idp.subject* in SamlTests
 f0e413cd7 MB-56517 Add stats which are not computed rates
 415253590 MB-56440: Allow concurrent sample bucket imports
 a888e38a8 cluster_tests: Unspecified requirements should be ignored
 1bfc080b3 MB-55047 Add prometheus http service discovery
 624d5e08f Allow partial and multiple task updates
 0148afaf7 MB-56487: Rename api_test->cluster_tests in CMakeLists.txt
 1be9ba032 MB-56487: Rename api_tests to cluster_tests
 7c76b83f1 MB-56411: Add log and configuration file to sigar_port
 472292451 MB-55732: Fix menelaus_roles:params_version_test()
couchstore is at c392c9b, changes since last good build: 
 c392c9b MB-57298: Fix handling of couch_compact --dropdeletes
 aebbaa7 MB-57314: Revert ctypes.CDLL() winmode behaviour to pre- Python 3.8
 e74c757 couchscript: Add missing header for ntohll()
forestdb is at 7af87ef, changes since last good build: 
 7af87ef MB-57466: Read offset exceeds the file's current offset error
kv_engine is at e36344c, changes since last good build: 
 e36344ce0 MB-55357: Validate Marker's visible-snap-end at PassiveStream
 221c3d348 MB-49779: dcpdrain: Add support for true_with_seqno_advanced OSO support
 76f62a958 Add number of clients and items_in_transit stats
 fe075052d MB-56671: Fix raciness when scheduling ItemFreqDecayerTask
 fc3f9a8c8 Extend dumping of command extras
 8aa46267d Allow for missing audit_descriptors in non-prod builds
 9bb3a9671 MB-56672: Limit concurrency on various tasks
 4a57437f5 RangeScanCreate don't accept extras
 af5d44fdc Don't include "key" attribute if its empty
Switching indexing to unstable
indexing is at 8bbe9ab, changes since last good build: none
Switching plasma to unstable
plasma is at 40f4da5, changes since last good build: 
fatal: Invalid revision range 3f9738166f88d527b0a58d2fbad97f0ba0a5b176..HEAD

Switching nitro to unstable
nitro is at f6a9378, changes since last good build: none
Switching gometa to master
gometa is at 96e745a, changes since last good build: none
Switching testrunner to master
Submodule 'java_sdk_client' (https://github.com/couchbaselabs/java_sdk_client) registered for path 'java_sdk_client'
Submodule 'lib/capellaAPI' (https://github.com/couchbaselabs/CapellaRESTAPIs) registered for path 'lib/capellaAPI'
Submodule 'magma_loader/DocLoader' (https://github.com/couchbaselabs/DocLoader.git) registered for path 'magma_loader/DocLoader'
Cloning into '/opt/build/testrunner/java_sdk_client'...
Cloning into '/opt/build/testrunner/lib/capellaAPI'...
Cloning into '/opt/build/testrunner/magma_loader/DocLoader'...
Submodule path 'java_sdk_client': checked out 'de89b059ce28567dbac18afb032271a4eaa674ff'
Submodule path 'lib/capellaAPI': checked out '7fc6f86544c143368f1f3d7be1d0192fdea4751b'
Submodule path 'magma_loader/DocLoader': checked out '0f32672eea77681939c9b26d43f60bd4b3478936'
testrunner is at 48f7a9d, changes since last good build: none
Pulling in uncommitted change 190721 at refs/changes/21/190721/33
Total 13 (delta 4), reused 7 (delta 4)
[unstable 154bd601] MB-55937: Introduce auth for systemHandler requests
 Author: Dhananjay Kshirsagar 
 Date: Wed May 10 12:21:42 2023 +0530
 5 files changed, 68 insertions(+), 24 deletions(-)
Pulling in uncommitted change 191342 at refs/changes/42/191342/9
Total 8 (delta 5), reused 5 (delta 5)
[unstable fa5a6cb5] MB-57427: disable array index tests
 Author: Dhruvil Shah 
 Date: Mon May 22 10:24:11 2023 +0530
 2 files changed, 7 insertions(+), 1 deletion(-)
Pulling in uncommitted change 193825 at refs/changes/25/193825/1
Total 6 (delta 5), reused 6 (delta 5)
[unstable 1e0469ad] MB100: CIFail - Enable info level client logs after TestScheduleIndexBasic
 Author: Sai Krishna Teja Kommaraju 
 Date: Fri Jul 14 00:58:44 2023 +0530
 1 file changed, 3 insertions(+)
Pulling in uncommitted change 195232 at refs/changes/32/195232/12
Total 49 (delta 43), reused 49 (delta 43)
[unstable 2735f663] MB-58279 Populate alternateShardIds for indexes getting upgraded (And re-plan placement if necessary)
 Author: Varun Velamuri 
 Date: Thu Aug 10 15:38:43 2023 -0700
 4 files changed, 198 insertions(+), 17 deletions(-)
Pulling in uncommitted change 195279 at refs/changes/79/195279/2
Total 5 (delta 4), reused 5 (delta 4)
[unstable 216adbf6] MB-57825: Enable LSS flush buffer tuning
 Author: Saptarshi Sen 
 Date: Fri Aug 11 10:53:36 2023 -0700
 1 file changed, 6 insertions(+), 6 deletions(-)
Pulling in uncommitted change 196138 at refs/changes/38/196138/3
Total 5 (delta 1), reused 1 (delta 1)
[unstable 98af51db] MB-54819: Keep dcp_bucket_seqnos.readerMap and dcp_bucket_seqnos.buckets in sync
 Author: Dhananjay Kshirsagar 
 Date: Mon Aug 28 12:48:49 2023 +0530
 1 file changed, 5 insertions(+), 1 deletion(-)
Pulling in uncommitted change 196494 at refs/changes/94/196494/1
Total 56 (delta 48), reused 56 (delta 48)
[unstable 681dc519] Sleep for 30 seconds and retry rebalance
 Author: Varun Velamuri 
 Date: Thu Aug 31 20:32:44 2023 -0700
 1 file changed, 23 insertions(+), 4 deletions(-)
Pulling in uncommitted change 196039 at refs/changes/39/196039/4
Total 18 (delta 7), reused 7 (delta 7)
[unstable 91df505] MB-58392: Add/update flush buffer unit tests
 Author: Saptarshi Sen 
 Date: Thu Aug 24 15:55:02 2023 -0700
 3 files changed, 173 insertions(+), 81 deletions(-)
Pulling in uncommitted change 196295 at refs/changes/95/196295/2
[unstable 21ae151] MB-58392: Fix a timer panic on zero LSSSampleInterval
 Author: Saptarshi Sen 
 Date: Mon Aug 28 14:16:41 2023 -0700
 1 file changed, 1 insertion(+), 1 deletion(-)
Pulling in uncommitted change 196296 at refs/changes/96/196296/2
[unstable cea0b40] MB-57825: Do not exceed flush buffer limit on resumeIO for idleLSS
 Author: Saptarshi Sen 
 Date: Mon Aug 28 16:12:34 2023 -0700
 2 files changed, 108 insertions(+), 6 deletions(-)
Building community edition
Building cmakefiles and deps [CE]
Building main product [CE]
Build CE finished
BUILD_ENTERPRISE empty. Building enterprise edition
Building Enterprise Edition
Building cmakefiles and deps [EE]
Building main product [EE]
Build EE finished

Testing

Started testing at 2023/09/01 10:33:52
Testing mode: sanity,unit,functional,serverless,integration
Using storage type: plasma
Setting ulimit to 200000

Simple Test

Sep 01 10:35:33 suite_setUp (rebalance.rebalancein.RebalanceInTests) ... ok
Sep 01 10:40:23 rebalance_in_with_ops (rebalance.rebalancein.RebalanceInTests) ... ok
Sep 01 10:44:50 rebalance_in_with_ops (rebalance.rebalancein.RebalanceInTests) ... ok
Sep 01 10:49:22 b" 'stop_on_failure': 'True'}"
Sep 01 10:49:22 b'./testrunner -i b/resources/dev-4-nodes-xdcr.ini -p makefile=True,stop_on_failure=True,log_level=CRITICAL -t rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops,nodes_in=3,replicas=1,items=50000,bucket_storage=couchstore,bucket_size=100,eviction_policy=valueOnly,get-logs-cluster-run=True,doc_ops=create;update;delete'
Sep 01 10:49:22 b"{'nodes_in': '3', 'replicas': '1', 'items': '50000', 'bucket_storage': 'couchstore', 'bucket_size': '100', 'eviction_policy': 'valueOnly', 'get-logs-cluster-run': 'True', 'doc_ops': 'create;update;delete', 'ini': 'b/resources/dev-4-nodes-xdcr.ini', 'cluster_name': 'dev-4-nodes-xdcr', 'spec': 'simple', 'conf_file': 'conf/simple.conf', 'makefile': 'True', 'stop_on_failure': 'True', 'log_level': 'CRITICAL', 'num_nodes': 4, 'case_number': 1, 'total_testcases': 8, 'last_case_fail': 'False', 'teardown_run': 'False', 'logs_folder': '/opt/build/testrunner/logs/testrunner-23-Sep-01_10-34-16/test_1'}"
Sep 01 10:49:22 b'-->result: '
Sep 01 10:49:22 b'summary so far suite rebalance.rebalancein.RebalanceInTests , pass 1 , fail 0'
Sep 01 10:49:22 b'./testrunner -i b/resources/dev-4-nodes-xdcr.ini -p makefile=True,stop_on_failure=True,log_level=CRITICAL -t rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops,nodes_in=3,bucket_type=ephemeral,replicas=1,items=50000,get-logs-cluster-run=True,doc_ops=create;update;delete'
Sep 01 10:49:22 b"{'nodes_in': '3', 'bucket_type': 'ephemeral', 'replicas': '1', 'items': '50000', 'get-logs-cluster-run': 'True', 'doc_ops': 'create;update;delete', 'ini': 'b/resources/dev-4-nodes-xdcr.ini', 'cluster_name': 'dev-4-nodes-xdcr', 'spec': 'simple', 'conf_file': 'conf/simple.conf', 'makefile': 'True', 'stop_on_failure': 'True', 'log_level': 'CRITICAL', 'num_nodes': 4, 'case_number': 2, 'total_testcases': 8, 'last_case_fail': 'False', 'teardown_run': 'True', 'logs_folder': '/opt/build/testrunner/logs/testrunner-23-Sep-01_10-34-16/test_2'}"
Sep 01 10:49:22 b'summary so far suite rebalance.rebalancein.RebalanceInTests , pass 2 , fail 0'
Sep 01 10:49:22 b'./testrunner -i b/resources/dev-4-nodes-xdcr.ini -p makefile=True,stop_on_failure=True,log_level=CRITICAL -t memcapable.WarmUpMemcachedTest.do_warmup_100k,bucket_storage=couchstore,bucket_size=100,eviction_policy=valueOnly,get-logs-cluster-run=True'
Sep 01 10:49:22 b"{'bucket_storage': 'couchstore', 'bucket_size': '100', 'eviction_policy': 'valueOnly', 'get-logs-cluster-run': 'True', 'ini': 'b/resources/dev-4-nodes-xdcr.ini', 'cluster_name': 'dev-4-nodes-xdcr', 'spec': 'simple', 'conf_file': 'conf/simple.conf', 'makefile': 'True', 'stop_on_failure': 'True', 'log_level': 'CRITICAL', 'num_nodes': 4, 'case_number': 3, 'total_testcases': 8, 'last_case_fail': 'False', 'teardown_run': 'True', 'logs_folder': '/opt/build/testrunner/logs/testrunner-23-Sep-01_10-34-16/test_3'}"
Sep 01 10:49:22 b'summary so far suite rebalance.rebalancein.RebalanceInTests , pass 2 , fail 0'
Sep 01 10:49:22 b'summary so far suite memcapable.WarmUpMemcachedTest , pass 0 , fail 1'
Sep 01 10:49:22 b'failures so far...'
Sep 01 10:49:22 b'./testrunner -i b/resources/dev-4-nodes-xdcr.ini -p makefile=True,stop_on_failure=True,log_level=CRITICAL -t view.createdeleteview.CreateDeleteViewTests.test_view_ops,ddoc_ops=create,test_with_view=True,num_ddocs=1,bucket_storage=couchstore,bucket_size=100,eviction_policy=valueOnly,num_views_per_ddoc=10,items=1000,skip_cleanup=False,get-logs-cluster-run=True'
Sep 01 10:49:22 b"{'ddoc_ops': 'create', 'test_with_view': 'True', 'num_ddocs': '1', 'bucket_storage': 'couchstore', 'bucket_size': '100', 'eviction_policy': 'valueOnly', 'num_views_per_ddoc': '10', 'items': '1000', 'skip_cleanup': 'False', 'get-logs-cluster-run': 'True', 'ini': 'b/resources/dev-4-nodes-xdcr.ini', 'cluster_name': 'dev-4-nodes-xdcr', 'spec': 'simple', 'conf_file': 'conf/simple.conf', 'makefile': 'True', 'stop_on_failure': 'True', 'log_level': 'CRITICAL', 'num_nodes': 4, 'case_number': 4, 'total_testcases': 8, 'last_case_fail': 'True', 'teardown_run': 'True', 'logs_folder': '/opt/build/testrunner/logs/testrunner-23-Sep-01_10-34-16/test_4'}"
Sep 01 10:50:31 b'http://127.0.0.1:9003/pools/default with status False: unknown pool'ok
Sep 01 10:52:53     self.failures.append((test, self._exc_info_to_string(err, test)))
Sep 01 10:52:53     self.failures.append((test, self._exc_info_to_string(err, test)))
Sep 01 10:52:53     self.failures.append((test, self._exc_info_to_string(err, test)))
Sep 01 10:52:53     self.failures.append((test, self._exc_info_to_string(err, test)))
Sep 01 10:52:53     self.failures.append((test, self._exc_info_to_string(err, test)))
Sep 01 10:52:53     self.failures.append((test, self._exc_info_to_string(err, test)))
Sep 01 10:52:53     self.fail(self._form_report_failure(failures, views))
Sep 01 10:52:53     self.failures.append((test, self._exc_info_to_string(err, test)))
Sep 01 10:52:59 FAILED (failures=1)
Sep 01 10:53:20     self.fail(ex)
Sep 01 10:53:20 FAILED (failures=1)
Sep 01 10:58:13 b'summary so far suite rebalance.rebalancein.RebalanceInTests , pass 2 , fail 0'
Sep 01 10:58:13 b'summary so far suite memcapable.WarmUpMemcachedTest , pass 0 , fail 1'
Sep 01 10:58:13 b'failures so far...'
Sep 01 10:58:13 b'summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0'
Sep 01 10:58:13 b'./testrunner -i b/resources/dev-4-nodes-xdcr.ini -p makefile=True,stop_on_failure=True,log_level=CRITICAL -t view.viewquerytests.ViewQueryTests.test_employee_dataset_startkey_endkey_queries_rebalance_in,num_nodes_to_add=1,bucket_storage=couchstore,bucket_size=100,eviction_policy=valueOnly,skip_rebalance=true,docs-per-day=1,timeout=1200,get-logs-cluster-run=True'
Sep 01 10:58:13 b"{'num_nodes_to_add': '1', 'bucket_storage': 'couchstore', 'bucket_size': '100', 'eviction_policy': 'valueOnly', 'skip_rebalance': 'true', 'docs-per-day': '1', 'timeout': '1200', 'get-logs-cluster-run': 'True', 'ini': 'b/resources/dev-4-nodes-xdcr.ini', 'cluster_name': 'dev-4-nodes-xdcr', 'spec': 'simple', 'conf_file': 'conf/simple.conf', 'makefile': 'True', 'stop_on_failure': 'True', 'log_level': 'CRITICAL', 'num_nodes': 4, 'case_number': 5, 'total_testcases': 8, 'last_case_fail': 'False', 'teardown_run': 'True', 'logs_folder': '/opt/build/testrunner/logs/testrunner-23-Sep-01_10-34-16/test_5'}"
Sep 01 10:58:13 b'summary so far suite rebalance.rebalancein.RebalanceInTests , pass 2 , fail 0'
Sep 01 10:58:13 b'summary so far suite memcapable.WarmUpMemcachedTest , pass 0 , fail 1'
Sep 01 10:58:13 b'failures so far...'
Sep 01 10:58:13 b'summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0'
Sep 01 10:58:13 b'summary so far suite view.viewquerytests.ViewQueryTests , pass 0 , fail 1'
Sep 01 10:58:13 b'failures so far...'
Sep 01 10:58:13 b'./testrunner -i b/resources/dev-4-nodes-xdcr.ini -p makefile=True,stop_on_failure=True,log_level=CRITICAL -t view.viewquerytests.ViewQueryTests.test_simple_dataset_stale_queries_data_modification,num-docs=1000,bucket_storage=couchstore,bucket_size=100,eviction_policy=valueOnly,skip_rebalance=true,timeout=1200,get-logs-cluster-run=True'
Sep 01 10:58:13 b"{'num-docs': '1000', 'bucket_storage': 'couchstore', 'bucket_size': '100', 'eviction_policy': 'valueOnly', 'skip_rebalance': 'true', 'timeout': '1200', 'get-logs-cluster-run': 'True', 'ini': 'b/resources/dev-4-nodes-xdcr.ini', 'cluster_name': 'dev-4-nodes-xdcr', 'spec': 'simple', 'conf_file': 'conf/simple.conf', 'makefile': 'True', 'stop_on_failure': 'True', 'log_level': 'CRITICAL', 'num_nodes': 4, 'case_number': 6, 'total_testcases': 8, 'last_case_fail': 'True', 'teardown_run': 'True', 'logs_folder': '/opt/build/testrunner/logs/testrunner-23-Sep-01_10-34-16/test_6'}"
Sep 01 10:58:13 b'summary so far suite rebalance.rebalancein.RebalanceInTests , pass 2 , fail 0'
Sep 01 10:58:13 b'summary so far suite memcapable.WarmUpMemcachedTest , pass 0 , fail 1'
Sep 01 10:58:13 b'failures so far...'
Sep 01 10:58:13 b'summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0'
Sep 01 10:58:13 b'summary so far suite view.viewquerytests.ViewQueryTests , pass 0 , fail 2'
Sep 01 10:58:13 b'failures so far...'
Sep 01 10:58:13 b'./testrunner -i b/resources/dev-4-nodes-xdcr.ini -p makefile=True,stop_on_failure=True,log_level=CRITICAL -t xdcr.uniXDCR.unidirectional.load_with_ops,replicas=1,items=10000,value_size=128,ctopology=chain,bucket_storage=couchstore,bucket_size=100,eviction_policy=valueOnly,rdirection=unidirection,doc-ops=update-delete,get-logs-cluster-run=True'
Sep 01 10:58:13 b"{'replicas': '1', 'items': '10000', 'value_size': '128', 'ctopology': 'chain', 'bucket_storage': 'couchstore', 'bucket_size': '100', 'eviction_policy': 'valueOnly', 'rdirection': 'unidirection', 'doc-ops': 'update-delete', 'get-logs-cluster-run': 'True', 'ini': 'b/resources/dev-4-nodes-xdcr.ini', 'cluster_name': 'dev-4-nodes-xdcr', 'spec': 'simple', 'conf_file': 'conf/simple.conf', 'makefile': 'True', 'stop_on_failure': 'True', 'log_level': 'CRITICAL', 'num_nodes': 4, 'case_number': 7, 'total_testcases': 8, 'last_case_fail': 'True', 'teardown_run': 'False', 'logs_folder': '/opt/build/testrunner/logs/testrunner-23-Sep-01_10-34-16/test_7'}"
Sep 01 10:58:13 b'summary so far suite rebalance.rebalancein.RebalanceInTests , pass 2 , fail 0'
Sep 01 10:58:13 b'summary so far suite memcapable.WarmUpMemcachedTest , pass 0 , fail 1'
Sep 01 10:58:13 b'failures so far...'
Sep 01 10:58:13 b'summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0'ok
Sep 01 11:02:32 load_with_failover (xdcr.uniXDCR.unidirectional) ... ok
Sep 01 11:05:25 suite_tearDown (xdcr.uniXDCR.unidirectional) ... ok
Sep 01 11:05:26 b'summary so far suite view.viewquerytests.ViewQueryTests , pass 0 , fail 2'
Sep 01 11:05:26 b'failures so far...'
Sep 01 11:05:26 b'summary so far suite xdcr.uniXDCR.unidirectional , pass 1 , fail 0'
Sep 01 11:05:26 b'./testrunner -i b/resources/dev-4-nodes-xdcr.ini -p makefile=True,stop_on_failure=True,log_level=CRITICAL -t xdcr.uniXDCR.unidirectional.load_with_failover,replicas=1,items=10000,ctopology=chain,rdirection=unidirection,bucket_storage=couchstore,bucket_size=100,eviction_policy=valueOnly,doc-ops=update-delete,failover=source,get-logs-cluster-run=True'
Sep 01 11:05:26 b"{'replicas': '1', 'items': '10000', 'ctopology': 'chain', 'rdirection': 'unidirection', 'bucket_storage': 'couchstore', 'bucket_size': '100', 'eviction_policy': 'valueOnly', 'doc-ops': 'update-delete', 'failover': 'source', 'get-logs-cluster-run': 'True', 'ini': 'b/resources/dev-4-nodes-xdcr.ini', 'cluster_name': 'dev-4-nodes-xdcr', 'spec': 'simple', 'conf_file': 'conf/simple.conf', 'makefile': 'True', 'stop_on_failure': 'True', 'log_level': 'CRITICAL', 'num_nodes': 4, 'case_number': 8, 'total_testcases': 8, 'last_case_fail': 'False', 'teardown_run': 'False', 'logs_folder': '/opt/build/testrunner/logs/testrunner-23-Sep-01_10-34-16/test_8'}"
Sep 01 11:05:26 b'summary so far suite rebalance.rebalancein.RebalanceInTests , pass 2 , fail 0'
Sep 01 11:05:26 b'summary so far suite memcapable.WarmUpMemcachedTest , pass 0 , fail 1'
Sep 01 11:05:26 b'failures so far...'
Sep 01 11:05:26 b'summary so far suite view.createdeleteview.CreateDeleteViewTests , pass 1 , fail 0'
Sep 01 11:05:26 b'summary so far suite view.viewquerytests.ViewQueryTests , pass 0 , fail 2'
Sep 01 11:05:26 b'failures so far...'
Sep 01 11:05:26 b'summary so far suite xdcr.uniXDCR.unidirectional , pass 2 , fail 0'
Sep 01 11:05:26 b'Run after suite setup for xdcr.uniXDCR.unidirectional.load_with_failover'
Sep 01 11:05:26 b"('rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops', ' pass')"
Sep 01 11:05:26 b"('rebalance.rebalancein.RebalanceInTests.rebalance_in_with_ops', ' pass')"
Sep 01 11:05:26 b"('memcapable.WarmUpMemcachedTest.do_warmup_100k', ' fail ')"
Sep 01 11:05:26 b"('view.createdeleteview.CreateDeleteViewTests.test_view_ops', ' pass')"
Sep 01 11:05:26 b"('view.viewquerytests.ViewQueryTests.test_employee_dataset_startkey_endkey_queries_rebalance_in', ' fail ')"
Sep 01 11:05:26 b"('view.viewquerytests.ViewQueryTests.test_simple_dataset_stale_queries_data_modification', ' fail ')"
Sep 01 11:05:26 b"('xdcr.uniXDCR.unidirectional.load_with_ops', ' pass')"
Sep 01 11:05:26 b"('xdcr.uniXDCR.unidirectional.load_with_failover', ' pass')"
Sep 01 11:05:26 Makefile:30: recipe for target 'simple-test' failed

Unit tests

=== RUN   TestMerger
--- PASS: TestMerger (0.02s)
=== RUN   TestInsert
--- PASS: TestInsert (0.00s)
=== RUN   TestInsertPerf
16000 items took 15.718042ms -> 1.017938493865839e+06 items/s conflicts 5
--- PASS: TestInsertPerf (0.02s)
=== RUN   TestGetPerf
16000 items took 8.665211ms -> 1.846463981084823e+06 items/s
--- PASS: TestGetPerf (0.01s)
=== RUN   TestGetRangeSplitItems
{
"node_count":             1000,
"soft_deletes":           0,
"read_conflicts":         0,
"insert_conflicts":       0,
"next_pointers_per_node": 1.3050,
"memory_used":            44880,
"node_allocs":            1000,
"node_frees":             0,
"level_node_distribution":{
"level0": 757,
"level1": 195,
"level2": 36,
"level3": 10,
"level4": 2,
"level5": 0,
"level6": 0,
"level7": 0,
"level8": 0,
"level9": 0,
"level10": 0,
"level11": 0,
"level12": 0,
"level13": 0,
"level14": 0,
"level15": 0,
"level16": 0,
"level17": 0,
"level18": 0,
"level19": 0,
"level20": 0,
"level21": 0,
"level22": 0,
"level23": 0,
"level24": 0,
"level25": 0,
"level26": 0,
"level27": 0,
"level28": 0,
"level29": 0,
"level30": 0,
"level31": 0,
"level32": 0
}
}
Split range keys [77 137 286 423 696 753]
No of items in each range [77 60 149 137 273 57 247]
--- PASS: TestGetRangeSplitItems (0.00s)
=== RUN   TestBuilder
{
"node_count":             50000,
"soft_deletes":           0,
"read_conflicts":         0,
"insert_conflicts":       0,
"next_pointers_per_node": 1.3329,
"memory_used":            2266320,
"node_allocs":            50000,
"node_frees":             0,
"level_node_distribution":{
"level0": 37419,
"level1": 9540,
"level2": 2298,
"level3": 543,
"level4": 140,
"level5": 44,
"level6": 13,
"level7": 2,
"level8": 1,
"level9": 0,
"level10": 0,
"level11": 0,
"level12": 0,
"level13": 0,
"level14": 0,
"level15": 0,
"level16": 0,
"level17": 0,
"level18": 0,
"level19": 0,
"level20": 0,
"level21": 0,
"level22": 0,
"level23": 0,
"level24": 0,
"level25": 0,
"level26": 0,
"level27": 0,
"level28": 0,
"level29": 0,
"level30": 0,
"level31": 0,
"level32": 0
}
}
Took 9.139535ms to build 50000 items, 5.470738e+06 items/sec
Took 1.047647ms to iterate 50000 items
--- PASS: TestBuilder (0.01s)
=== RUN   TestNodeDCAS
--- PASS: TestNodeDCAS (0.00s)
PASS
ok  	github.com/couchbase/nitro/skiplist	0.074s
=== RUN   TestZstdSimple
--- PASS: TestZstdSimple (0.00s)
=== RUN   TestZstdCompressBound
--- PASS: TestZstdCompressBound (3.06s)
=== RUN   TestZstdErrors
--- PASS: TestZstdErrors (0.00s)
=== RUN   TestZstdCompressLevels
--- PASS: TestZstdCompressLevels (0.76s)
=== RUN   TestZstdEmptySrc
--- PASS: TestZstdEmptySrc (0.00s)
=== RUN   TestZstdLargeSrc
--- PASS: TestZstdLargeSrc (0.00s)
PASS
ok  	github.com/couchbase/plasma/zstd	3.830s
go: downloading github.com/aws/aws-sdk-go-v2 v1.18.0
go: downloading github.com/aws/aws-sdk-go-v2/config v1.18.22
go: downloading github.com/aws/aws-sdk-go-v2/service/s3 v1.33.0
go: downloading github.com/aws/smithy-go v1.13.5
go: downloading github.com/aws/aws-sdk-go-v2/service/sts v1.18.10
go: downloading github.com/aws/aws-sdk-go-v2/credentials v1.13.21
go: downloading github.com/aws/aws-sdk-go-v2/internal/ini v1.3.34
go: downloading github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.3
go: downloading github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.9
go: downloading github.com/aws/aws-sdk-go-v2/service/sso v1.12.9
go: downloading github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.9.11
go: downloading github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.10
go: downloading github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.33
go: downloading github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.1.28
go: downloading github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.27
go: downloading github.com/aws/aws-sdk-go-v2/internal/v4a v1.0.25
go: downloading github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.14.2
go: downloading github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.27
=== RUN   TestAutoTunerWriteUsageStats
--- PASS: TestAutoTunerWriteUsageStats (12.13s)
=== RUN   TestAutoTunerReadUsageStats
--- PASS: TestAutoTunerReadUsageStats (8.69s)
=== RUN   TestAutoTunerCleanerUsageStats
--- PASS: TestAutoTunerCleanerUsageStats (9.31s)
=== RUN   TestAutoTunerDiskStats
--- PASS: TestAutoTunerDiskStats (2.50s)
=== RUN   TestAutoTunerTargetFragRatio
--- PASS: TestAutoTunerTargetFragRatio (0.00s)
=== RUN   TestAutoTunerExcessUsedSpace
--- PASS: TestAutoTunerExcessUsedSpace (0.00s)
=== RUN   TestAutoTunerUsedSpaceRatio
--- PASS: TestAutoTunerUsedSpaceRatio (0.00s)
=== RUN   TestAutoTunerAdjustFragRatio
--- PASS: TestAutoTunerAdjustFragRatio (0.00s)
=== RUN   TestAutoTuneFlushBufferAdjustMemQuotaSingleShard
--- PASS: TestAutoTuneFlushBufferAdjustMemQuotaSingleShard (16.47s)
=== RUN   TestAutoTuneFlushBufferAdjustMemQuotaManyShards
--- PASS: TestAutoTuneFlushBufferAdjustMemQuotaManyShards (17.86s)
=== RUN   TestAutoTuneFlushBufferRebalanceIdleShards
--- PASS: TestAutoTuneFlushBufferRebalanceIdleShards (15.36s)
=== RUN   TestAutoTuneFlushBufferGetUsedMemory
--- PASS: TestAutoTuneFlushBufferGetUsedMemory (28.24s)
=== RUN   TestAutoTuneFlushBufferIdleResume
--- PASS: TestAutoTuneFlushBufferIdleResume (1.60s)
=== RUN   TestBloom
--- PASS: TestBloom (6.62s)
=== RUN   TestBloomDisableEnable
--- PASS: TestBloomDisableEnable (3.95s)
=== RUN   TestBloomDisable
--- PASS: TestBloomDisable (0.07s)
=== RUN   TestBloomFreeDuringLookup
--- PASS: TestBloomFreeDuringLookup (0.06s)
=== RUN   TestBloomRecoveryFreeDuringLookup
--- PASS: TestBloomRecoveryFreeDuringLookup (0.08s)
=== RUN   TestBloomRecoverySwapInLookup
--- PASS: TestBloomRecoverySwapInLookup (0.11s)
=== RUN   TestBloomRecoverySwapOutLookup
--- PASS: TestBloomRecoverySwapOutLookup (0.09s)
=== RUN   TestBloomRecoveryInserts
--- PASS: TestBloomRecoveryInserts (0.11s)
=== RUN   TestBloomRecovery
--- PASS: TestBloomRecovery (0.15s)
=== RUN   TestBloomStats
--- PASS: TestBloomStats (3.94s)
=== RUN   TestBloomStatsRecovery
--- PASS: TestBloomStatsRecovery (1.38s)
=== RUN   TestBloomFilterSimple
--- PASS: TestBloomFilterSimple (0.00s)
=== RUN   TestBloomFilterConcurrent
--- PASS: TestBloomFilterConcurrent (23.13s)
=== RUN   TestBitArrayConcurrent
--- PASS: TestBitArrayConcurrent (0.91s)
=== RUN   TestBloomCapacity
--- PASS: TestBloomCapacity (0.00s)
=== RUN   TestBloomNumHashFuncs
--- PASS: TestBloomNumHashFuncs (0.00s)
=== RUN   TestBloomTestAndAdd
--- PASS: TestBloomTestAndAdd (0.22s)
=== RUN   TestBloomReset
--- PASS: TestBloomReset (0.00s)
=== RUN   TestLFSCopier
--- PASS: TestLFSCopier (0.00s)
=== RUN   TestNodeCopier
--- PASS: TestNodeCopier (0.03s)
=== RUN   TestLFSCopierNumBytes
--- PASS: TestLFSCopierNumBytes (0.01s)
=== RUN   TestNodeCopierNumBytes
--- PASS: TestNodeCopierNumBytes (0.05s)
=== RUN   TestCopierRestoreFile
--- PASS: TestCopierRestoreFile (0.00s)
=== RUN   TestCopierUploadDownloadBytes
--- PASS: TestCopierUploadDownloadBytes (0.02s)
=== RUN   TestCopierKeyEncoding
--- PASS: TestCopierKeyEncoding (0.23s)
=== RUN   TestS3CopierWithRetry
--- PASS: TestS3CopierWithRetry (0.00s)
=== RUN   TestS3RateLimitUnsignedPayload
--- PASS: TestS3RateLimitUnsignedPayload (0.00s)
=== RUN   TestSBCopyConcurrent
--- PASS: TestSBCopyConcurrent (1.34s)
=== RUN   TestSBCopyCorrupt
--- PASS: TestSBCopyCorrupt (0.01s)
=== RUN   TestLSSCopyHeadTailSingleSegment
--- PASS: TestLSSCopyHeadTailSingleSegment (0.06s)
=== RUN   TestLSSCopyFullSegments
--- PASS: TestLSSCopyFullSegments (0.79s)
=== RUN   TestLSSCopyPartialSegments
--- PASS: TestLSSCopyPartialSegments (1.13s)
=== RUN   TestLSSCopyHolePunching
--- PASS: TestLSSCopyHolePunching (0.71s)
=== RUN   TestLSSCopyConcurrent
--- PASS: TestLSSCopyConcurrent (0.94s)
=== RUN   TestLSSCopyHeapUsage
--- PASS: TestLSSCopyHeapUsage (19.08s)
=== RUN   TestLSSCopyWithError
--- PASS: TestLSSCopyWithError (0.00s)
=== RUN   TestRateLimitReaderSeekOff
--- PASS: TestRateLimitReaderSeekOff (11.80s)
=== RUN   TestRateLimitMaxIOPS
--- PASS: TestRateLimitMaxIOPS (15.46s)
=== RUN   TestNodeRateLimiterUpdate
--- PASS: TestNodeRateLimiterUpdate (0.00s)
=== RUN   TestNodeRateLimiterMaxIOPs
--- PASS: TestNodeRateLimiterMaxIOPs (78.17s)
=== RUN   TestRateLimitConcurrentLSSCopy
--- PASS: TestRateLimitConcurrentLSSCopy (17.90s)
=== RUN   TestRateLimitDynamicConcurrentLSSCopy
--- PASS: TestRateLimitDynamicConcurrentLSSCopy (13.38s)
=== RUN   TestShardCopySimple
--- PASS: TestShardCopySimple (0.57s)
=== RUN   TestShardCopyWithCancel
--- PASS: TestShardCopyWithCancel (10.75s)
=== RUN   TestShardCopyNoRP
--- PASS: TestShardCopyNoRP (0.29s)
=== RUN   TestShardCopyMetadataCorrupted
--- PASS: TestShardCopyMetadataCorrupted (0.05s)
=== RUN   TestShardCopyMetadataCorruptedII
--- PASS: TestShardCopyMetadataCorruptedII (0.17s)
=== RUN   TestShardCopyLSSMetadataCorrupted
--- PASS: TestShardCopyLSSMetadataCorrupted (0.12s)
=== RUN   TestShardCopyBeforeRecovery
--- PASS: TestShardCopyBeforeRecovery (0.00s)
=== RUN   TestShardCopySkipLog
--- PASS: TestShardCopySkipLog (1.26s)
=== RUN   TestShardCopyAddInstance
--- PASS: TestShardCopyAddInstance (2.39s)
=== RUN   TestShardCopyDestroyInstance
--- PASS: TestShardCopyDestroyInstance (0.47s)
=== RUN   TestShardCopyRestoreShard
--- PASS: TestShardCopyRestoreShard (18.74s)
=== RUN   TestShardCopyRestoreManyShards
--- PASS: TestShardCopyRestoreManyShards (10.81s)
=== RUN   TestShardCopyRestoreConcurrentLogCleaning
--- PASS: TestShardCopyRestoreConcurrentLogCleaning (21.38s)
=== RUN   TestShardCopyRestorePartialRollback
--- PASS: TestShardCopyRestorePartialRollback (19.76s)
=== RUN   TestInvalidMVCCRollback
--- PASS: TestInvalidMVCCRollback (0.35s)
=== RUN   TestShardCopyRestoreConcurrentPurges
--- PASS: TestShardCopyRestoreConcurrentPurges (20.38s)
=== RUN   TestShardCopyRestoreRepairHdr
--- PASS: TestShardCopyRestoreRepairHdr (210.31s)
=== RUN   TestShardCopyDuplicateIndex
--- PASS: TestShardCopyDuplicateIndex (0.30s)
=== RUN   TestShardCopyEmptyIndex
--- PASS: TestShardCopyEmptyIndex (0.22s)
=== RUN   TestTenantCopy
--- PASS: TestTenantCopy (6.76s)
=== RUN   TestLockShardAddInstance
--- PASS: TestLockShardAddInstance (0.32s)
=== RUN   TestLockShardAddInstanceMapping
--- PASS: TestLockShardAddInstanceMapping (0.28s)
=== RUN   TestLockShardCloseInstance
--- PASS: TestLockShardCloseInstance (0.46s)
=== RUN   TestLockShardEmptyShard
--- PASS: TestLockShardEmptyShard (0.20s)
=== RUN   TestLockShardUseShardId
--- PASS: TestLockShardUseShardId (0.28s)
=== RUN   TestLockShardUseShardIdII
--- PASS: TestLockShardUseShardIdII (0.06s)
=== RUN   TestDestroyShardID
--- PASS: TestDestroyShardID (1.24s)
=== RUN   TestDestroyShardIDConcurrent
--- PASS: TestDestroyShardIDConcurrent (0.20s)
=== RUN   TestDestroyShardIDNumTenants
--- PASS: TestDestroyShardIDNumTenants (0.58s)
=== RUN   TestDestroyShardIDTenantAddRemove
--- PASS: TestDestroyShardIDTenantAddRemove (0.22s)
=== RUN   TestDestroyShardIDOnRestoreFailure
--- PASS: TestDestroyShardIDOnRestoreFailure (0.72s)
=== RUN   TestDestroyShardIDOnRestoreFailure2
--- PASS: TestDestroyShardIDOnRestoreFailure2 (1.01s)
=== RUN   TestTransferShardAPI
--- PASS: TestTransferShardAPI (285.15s)
=== RUN   TestTransferShardAPICreateIndexes
--- PASS: TestTransferShardAPICreateIndexes (19.46s)
=== RUN   TestTransferShardAPIWithDropIndexes
--- PASS: TestTransferShardAPIWithDropIndexes (6.31s)
=== RUN   TestTransferShardAPIWithCancel
--- PASS: TestTransferShardAPIWithCancel (6.88s)
=== RUN   TestTransferShardAPIWithCleanup
--- PASS: TestTransferShardAPIWithCleanup (89.53s)
=== RUN   TestRestoreShardAPI
--- PASS: TestRestoreShardAPI (6.64s)
=== RUN   TestRestoreShardNumShards
--- PASS: TestRestoreShardNumShards (0.37s)
=== RUN   TestRestoreShardInvalidLocation
--- PASS: TestRestoreShardInvalidLocation (0.11s)
=== RUN   TestRestoreShardReplicaRepair
--- PASS: TestRestoreShardReplicaRepair (19.79s)
=== RUN   TestRestoreShardDone
--- PASS: TestRestoreShardDone (0.97s)
=== RUN   TestRestoreShardDoneConcurrent
--- PASS: TestRestoreShardDoneConcurrent (1.76s)
=== RUN   TestRestoreShardDoneDroppedInstances
--- PASS: TestRestoreShardDoneDroppedInstances (1.28s)
=== RUN   TestShardDoCleanupAPI
--- PASS: TestShardDoCleanupAPI (1.25s)
=== RUN   TestShardDoCleanupAPI2
--- PASS: TestShardDoCleanupAPI2 (25.67s)
=== RUN   TestRestoreShardPauseResume
--- PASS: TestRestoreShardPauseResume (13.68s)
=== RUN   TestRateLimitConcurrentTransferRestoreShard
--- PASS: TestRateLimitConcurrentTransferRestoreShard (392.29s)
=== RUN   TestDiag
--- PASS: TestDiag (0.74s)
=== RUN   TestDumpLog
--- PASS: TestDumpLog (0.07s)
=== RUN   TestExtrasN1
=== RUN   TestExtrasN2
=== RUN   TestExtrasN3
=== RUN   TestGMRecovery
--- PASS: TestGMRecovery (13.79s)
=== RUN   TestIteratorSimple
--- PASS: TestIteratorSimple (4.89s)
=== RUN   TestIteratorSeek
--- PASS: TestIteratorSeek (5.99s)
=== RUN   TestPlasmaIteratorSeekFirst
--- PASS: TestPlasmaIteratorSeekFirst (0.52s)
=== RUN   TestPlasmaIteratorSwapin
--- PASS: TestPlasmaIteratorSwapin (5.10s)
=== RUN   TestIteratorSetEnd
--- PASS: TestIteratorSetEnd (0.76s)
=== RUN   TestIterHiItm
--- PASS: TestIterHiItm (2.78s)
=== RUN   TestIterDeleteSplitMerge
--- PASS: TestIterDeleteSplitMerge (0.04s)
=== RUN   TestKeySamplingSingle
--- PASS: TestKeySamplingSingle (0.14s)
=== RUN   TestKeySamplingAll
--- PASS: TestKeySamplingAll (0.17s)
=== RUN   TestKeySamplingEmpty
--- PASS: TestKeySamplingEmpty (0.04s)
=== RUN   TestKeySamplingExceed
--- PASS: TestKeySamplingExceed (0.15s)
=== RUN   TestLogOperation
--- PASS: TestLogOperation (60.41s)
=== RUN   TestLogLargeSize
--- PASS: TestLogLargeSize (0.17s)
=== RUN   TestLogTrim
--- PASS: TestLogTrim (60.27s)
=== RUN   TestLogSuperblockCorruption
--- PASS: TestLogSuperblockCorruption (60.12s)
=== RUN   TestLogTrimHolePunch
--- PASS: TestLogTrimHolePunch (50.07s)
=== RUN   TestLogMissingAndTruncatedSegments
--- PASS: TestLogMissingAndTruncatedSegments (0.08s)
=== RUN   TestLogReadBeyondMaxFileIndex
--- PASS: TestLogReadBeyondMaxFileIndex (2.60s)
=== RUN   TestLogReadEOFWithMMap
--- PASS: TestLogReadEOFWithMMap (0.01s)
=== RUN   TestLogLargeTrim
--- PASS: TestLogLargeTrim (0.01s)
=== RUN   TestLSSCleaningRewrite
--- PASS: TestLSSCleaningRewrite (0.70s)
=== RUN   TestShardLSSCleaning
--- PASS: TestShardLSSCleaning (0.27s)
=== RUN   TestShardLSSCleaningDeleteInstance
--- PASS: TestShardLSSCleaningDeleteInstance (0.27s)
=== RUN   TestShardLSSCleaningCorruptInstance
--- PASS: TestShardLSSCleaningCorruptInstance (0.24s)
=== RUN   TestPlasmaLSSCleaner
--- PASS: TestPlasmaLSSCleaner (218.72s)
=== RUN   TestPlasmaLSSCleanerMinSize
--- PASS: TestPlasmaLSSCleanerMinSize (12.25s)
=== RUN   TestSharedLSSCleanerAllDestroyedInstances
--- PASS: TestSharedLSSCleanerAllDestroyedInstances (259.40s)
=== RUN   TestLSSCleanerSyncWithConcurrentWriterRecovery
--- PASS: TestLSSCleanerSyncWithConcurrentWriterRecovery (259.08s)
=== RUN   TestLSSCleanerSyncWithConcurrentWriterRecoveryDGM
--- PASS: TestLSSCleanerSyncWithConcurrentWriterRecoveryDGM (285.65s)
=== RUN   TestLSSCleanerSyncWithMultiWriterRecovery
--- PASS: TestLSSCleanerSyncWithMultiWriterRecovery (111.91s)
=== RUN   TestLSSCleanerSyncWithMultiWriterRecoveryBigKeys
--- PASS: TestLSSCleanerSyncWithMultiWriterRecoveryBigKeys (87.47s)
=== RUN   TestLSSBasic
--- PASS: TestLSSBasic (0.09s)
=== RUN   TestLSSConcurrent
--- PASS: TestLSSConcurrent (0.86s)
=== RUN   TestLSSCleaner
--- PASS: TestLSSCleaner (12.75s)
=== RUN   TestLSSSuperBlock
--- PASS: TestLSSSuperBlock (1.13s)
=== RUN   TestLSSLargeSinglePayload
--- PASS: TestLSSLargeSinglePayload (0.80s)
=== RUN   TestLSSUnstableEnvironment
--- PASS: TestLSSUnstableEnvironment (10.24s)
=== RUN   TestLSSSmallFlushBuffer
--- PASS: TestLSSSmallFlushBuffer (0.01s)
=== RUN   TestLSSTrimFlushBufferGC
--- PASS: TestLSSTrimFlushBufferGC (1.56s)
=== RUN   TestLSSTrimFlushBufferNoIO
--- PASS: TestLSSTrimFlushBufferNoIO (10.01s)
=== RUN   TestLSSTrimFlushBufferSlowIO
--- PASS: TestLSSTrimFlushBufferSlowIO (5.03s)
=== RUN   TestLSSTrimFlushBufferWithIO
--- PASS: TestLSSTrimFlushBufferWithIO (17.33s)
=== RUN   TestLSSExtendFlushBufferWithIO
--- PASS: TestLSSExtendFlushBufferWithIO (62.01s)
=== RUN   TestLSSCtxTrimFlushBuffer
--- PASS: TestLSSCtxTrimFlushBuffer (5.47s)
=== RUN   TestLSSNegativeGetFlushBufferMemory
--- PASS: TestLSSNegativeGetFlushBufferMemory (0.02s)
=== RUN   TestLSSNegativeGetFlushBufferMemoryII
--- PASS: TestLSSNegativeGetFlushBufferMemoryII (0.03s)
=== RUN   TestLSSTrimFlushBufferSizeUnAligned
--- PASS: TestLSSTrimFlushBufferSizeUnAligned (0.00s)
=== RUN   TestMem
Plasma: Adaptive memory quota tuning (decrementing): RSS:7008800768, freePercent:52.83064087713639, currentQuota=1099511627776, newQuota=1073741824, netGrowth=0, percent=99, heap=0, active=0, alloc=0, stats=310, go=65314816Plasma: Adaptive memory quota tuning (incrementing): RSS:7008370688, freePercent: 52.83556658631935, currentQuota=0, newQuota=10995116277, heap=0, stats=310, go=64856064--- PASS: TestMem (1.52s)
=== RUN   TestCpu
--- PASS: TestCpu (14.49s)
=== RUN   TestTopTen20
--- PASS: TestTopTen20 (0.82s)
=== RUN   TestTopTen5
--- PASS: TestTopTen5 (0.19s)
=== RUN   TestMemAtLowResident
--- PASS: TestMemAtLowResident (3.58s)
=== RUN   TestMVCCSimple
--- PASS: TestMVCCSimple (0.28s)
=== RUN   TestMVCCLookup
--- PASS: TestMVCCLookup (0.17s)
=== RUN   TestMVCCIteratorRefresh
--- PASS: TestMVCCIteratorRefresh (5.33s)
=== RUN   TestMVCCIteratorRefreshEveryRow
--- PASS: TestMVCCIteratorRefreshEveryRow (1.29s)
=== RUN   TestMVCCGarbageCollection
--- PASS: TestMVCCGarbageCollection (0.14s)
=== RUN   TestMVCCRecoveryPoint
--- PASS: TestMVCCRecoveryPoint (2.82s)
=== RUN   TestMVCCRollbackMergeSibling
--- PASS: TestMVCCRollbackMergeSibling (0.05s)
=== RUN   TestMVCCRollbackCompact
--- PASS: TestMVCCRollbackCompact (0.10s)
=== RUN   TestMVCCRollbackSplit
--- PASS: TestMVCCRollbackSplit (0.05s)
=== RUN   TestMVCCRollbackItemsNotInSnapshot
--- PASS: TestMVCCRollbackItemsNotInSnapshot (0.21s)
=== RUN   TestMVCCRecoveryPointRollbackedSnapshot
--- PASS: TestMVCCRecoveryPointRollbackedSnapshot (1.40s)
=== RUN   TestMVCCRollbackBetweenRecoveryPoint
--- PASS: TestMVCCRollbackBetweenRecoveryPoint (1.41s)
=== RUN   TestMVCCRecoveryPointCrash
--- PASS: TestMVCCRecoveryPointCrash (0.11s)
=== RUN   TestMVCCIntervalGC
--- PASS: TestMVCCIntervalGC (0.28s)
=== RUN   TestMVCCItemsCount
--- PASS: TestMVCCItemsCount (0.48s)
=== RUN   TestLargeItems
--- PASS: TestLargeItems (105.55s)
=== RUN   TestTooLargeKey
--- PASS: TestTooLargeKey (3.49s)
=== RUN   TestMVCCItemUpdateSize
--- PASS: TestMVCCItemUpdateSize (0.33s)
=== RUN   TestEvictionStats
--- PASS: TestEvictionStats (0.70s)
=== RUN   TestReaderCacheStats
--- PASS: TestReaderCacheStats (1.17s)
=== RUN   TestInvalidSnapshot
--- PASS: TestInvalidSnapshot (1.40s)
=== RUN   TestEmptyKeyInsert
--- PASS: TestEmptyKeyInsert (0.03s)
=== RUN   TestMVCCRecoveryPointError
--- PASS: TestMVCCRecoveryPointError (0.05s)
=== RUN   TestMVCCReaderPurgeSequential
--- PASS: TestMVCCReaderPurgeSequential (0.32s)
=== RUN   TestMVCCReaderNoPurge
--- PASS: TestMVCCReaderNoPurge (0.31s)
=== RUN   TestMVCCReaderPurgeAfterUpdate
--- PASS: TestMVCCReaderPurgeAfterUpdate (0.30s)
=== RUN   TestMVCCReaderPurgeAfterRollback
--- PASS: TestMVCCReaderPurgeAfterRollback (0.35s)
=== RUN   TestMVCCReaderPurgeSimple
--- PASS: TestMVCCReaderPurgeSimple (0.06s)
=== RUN   TestMVCCReaderPurgeRandom
--- PASS: TestMVCCReaderPurgeRandom (0.29s)
=== RUN   TestMVCCReaderPurgePageFlag
--- PASS: TestMVCCReaderPurgePageFlag (0.18s)
=== RUN   TestMVCCPurgeRatioWithRollback
--- PASS: TestMVCCPurgeRatioWithRollback (16.36s)
=== RUN   TestComputeItemsCountMVCCWithRollbackI
--- PASS: TestComputeItemsCountMVCCWithRollbackI (0.08s)
=== RUN   TestComputeItemsCountMVCCWithRollbackII
--- PASS: TestComputeItemsCountMVCCWithRollbackII (0.10s)
=== RUN   TestComputeItemsCountMVCCWithRollbackIII
--- PASS: TestComputeItemsCountMVCCWithRollbackIII (0.05s)
=== RUN   TestComputeItemsCountMVCCWithRollbackIV
--- PASS: TestComputeItemsCountMVCCWithRollbackIV (0.12s)
=== RUN   TestMVCCPurgedRecordsWithCompactFullMarshalAndCascadedEmptyPagesMerge
--- PASS: TestMVCCPurgedRecordsWithCompactFullMarshalAndCascadedEmptyPagesMerge (2.79s)
=== RUN   TestMaxDeltaChainLenWithCascadedEmptyPagesMerge
--- PASS: TestMaxDeltaChainLenWithCascadedEmptyPagesMerge (2.52s)
=== RUN   TestAutoHoleCleaner
--- PASS: TestAutoHoleCleaner (49.73s)
=== RUN   TestAutoHoleCleaner5Indexes
--- PASS: TestAutoHoleCleaner5Indexes (283.10s)
=== RUN   TestIteratorReportedHoleRegionBoundary
--- PASS: TestIteratorReportedHoleRegionBoundary (0.14s)
=== RUN   TestFullRangeHoleScans
--- PASS: TestFullRangeHoleScans (0.54s)
=== RUN   TestOverlappingRangeHoleScans
--- PASS: TestOverlappingRangeHoleScans (0.51s)
=== RUN   TestMVCCIteratorSMRRefreshOnHoleScan
--- PASS: TestMVCCIteratorSMRRefreshOnHoleScan (12.49s)
=== RUN   TestAutoHoleCleanerWithRecovery
--- PASS: TestAutoHoleCleanerWithRecovery (2.47s)
=== RUN   TestPageMergeCorrectness2
--- PASS: TestPageMergeCorrectness2 (0.00s)
=== RUN   TestPageMergeCorrectness
--- PASS: TestPageMergeCorrectness (0.00s)
=== RUN   TestPageMarshalFull
--- PASS: TestPageMarshalFull (0.01s)
=== RUN   TestPageMergeMarshal
--- PASS: TestPageMergeMarshal (0.00s)
=== RUN   TestPageOperations
--- PASS: TestPageOperations (0.03s)
=== RUN   TestPageIterator
--- PASS: TestPageIterator (0.00s)
=== RUN   TestPageMarshal
--- PASS: TestPageMarshal (0.02s)
=== RUN   TestPageMergeCorrectness3
--- PASS: TestPageMergeCorrectness3 (0.00s)
=== RUN   TestPageHasDataRecords
--- PASS: TestPageHasDataRecords (0.00s)
=== RUN   TestPlasmaPageVisitor
--- PASS: TestPlasmaPageVisitor (4.60s)
=== RUN   TestPageRingVisitor
--- PASS: TestPageRingVisitor (4.43s)
=== RUN   TestPauseVisitorOnLowMemory
--- PASS: TestPauseVisitorOnLowMemory (1.17s)
=== RUN   TestCheckpointRecovery
--- PASS: TestCheckpointRecovery (13.03s)
=== RUN   TestPageCorruption
--- PASS: TestPageCorruption (1.34s)
=== RUN   TestCheckPointRecoveryFollowCleaning
--- PASS: TestCheckPointRecoveryFollowCleaning (0.10s)
=== RUN   TestFragmentationWithZeroItems
--- PASS: TestFragmentationWithZeroItems (1.17s)
=== RUN   TestEvictOnPersist
--- PASS: TestEvictOnPersist (0.24s)
=== RUN   TestPlasmaSimple
--- PASS: TestPlasmaSimple (13.80s)
=== RUN   TestPlasmaCompression
--- PASS: TestPlasmaCompression (0.03s)
=== RUN   TestPlasmaCompressionWrong
--- PASS: TestPlasmaCompressionWrong (0.03s)
=== RUN   TestPlasmaInMemCompression
--- PASS: TestPlasmaInMemCompression (0.04s)
=== RUN   TestPlasmaInMemCompressionZstd
--- PASS: TestPlasmaInMemCompressionZstd (0.05s)
=== RUN   TestPlasmaInMemCompressionWrong
--- PASS: TestPlasmaInMemCompressionWrong (0.04s)
=== RUN   TestSpoiledConfig
--- PASS: TestSpoiledConfig (0.04s)
=== RUN   TestPlasmaErrorFile
--- PASS: TestPlasmaErrorFile (0.03s)
=== RUN   TestPlasmaPersistor
--- PASS: TestPlasmaPersistor (9.88s)
=== RUN   TestPlasmaEvictionLSSDataSize
--- PASS: TestPlasmaEvictionLSSDataSize (0.07s)
=== RUN   TestPlasmaEviction
--- PASS: TestPlasmaEviction (29.71s)
=== RUN   TestConcurrDelOps
--- PASS: TestConcurrDelOps (72.68s)
=== RUN   TestPlasmaDataSize
--- PASS: TestPlasmaDataSize (0.04s)
=== RUN   TestLargeBasePage
--- PASS: TestLargeBasePage (40.26s)
=== RUN   TestLargeValue
--- PASS: TestLargeValue (94.74s)
=== RUN   TestPlasmaTooLargeKey
--- PASS: TestPlasmaTooLargeKey (3.49s)
=== RUN   TestEvictAfterMerge
--- PASS: TestEvictAfterMerge (0.16s)
=== RUN   TestEvictDirty
--- PASS: TestEvictDirty (0.22s)
=== RUN   TestEvictUnderQuota
--- PASS: TestEvictUnderQuota (60.17s)
=== RUN   TestEvictSetting
--- PASS: TestEvictSetting (1.28s)
=== RUN   TestBasePageAfterCompaction
--- PASS: TestBasePageAfterCompaction (0.17s)
=== RUN   TestSwapout
--- PASS: TestSwapout (0.03s)
=== RUN   TestSwapoutSplitBasePage
--- PASS: TestSwapoutSplitBasePage (0.06s)
=== RUN   TestCompactFullMarshal
--- PASS: TestCompactFullMarshal (0.07s)
=== RUN   TestPageStats
--- PASS: TestPageStats (3.09s)
=== RUN   TestPageStatsTinyIndex
--- PASS: TestPageStatsTinyIndex (0.15s)
=== RUN   TestPageStatsTinyIndexOnRecovery
--- PASS: TestPageStatsTinyIndexOnRecovery (0.10s)
=== RUN   TestPageStatsTinyIndexOnSplitAndMerge
--- PASS: TestPageStatsTinyIndexOnSplitAndMerge (0.06s)
=== RUN   TestPageCompress
--- PASS: TestPageCompress (0.06s)
=== RUN   TestPageCompressSwapin
--- PASS: TestPageCompressSwapin (0.05s)
=== RUN   TestPageCompressStats
--- PASS: TestPageCompressStats (1.17s)
=== RUN   TestPageDecompressStats
--- PASS: TestPageDecompressStats (0.04s)
=== RUN   TestSharedDedicatedDataSize
--- PASS: TestSharedDedicatedDataSize (5.27s)
=== RUN   TestLastRpSns
--- PASS: TestLastRpSns (0.06s)
=== RUN   TestPageCompressState
--- PASS: TestPageCompressState (0.06s)
=== RUN   TestPageCompressDuringBurst
--- PASS: TestPageCompressDuringBurst (0.07s)
=== RUN   TestPageDontDecompressDuringScan
--- PASS: TestPageDontDecompressDuringScan (0.16s)
=== RUN   TestPageDecompressAndCompressSwapin
--- PASS: TestPageDecompressAndCompressSwapin (2.06s)
=== RUN   TestPageCompressibleStat
--- PASS: TestPageCompressibleStat (0.57s)
=== RUN   TestPageCompressibleStatRecovery
--- PASS: TestPageCompressibleStatRecovery (0.18s)
=== RUN   TestPageCompressBeforeEvictPercent
--- PASS: TestPageCompressBeforeEvictPercent (1.22s)
=== RUN   TestPageCompressDecompressAfterDisable
--- PASS: TestPageCompressDecompressAfterDisable (1.22s)
=== RUN   TestPageKeepSwapinChainCompressed
--- PASS: TestPageKeepSwapinChainCompressed (0.06s)
=== RUN   TestEnforceSameCompressionAlgo
--- PASS: TestEnforceSameCompressionAlgo (0.26s)
=== RUN   TestPageCompressChangeAlgo
--- PASS: TestPageCompressChangeAlgo (0.77s)
=== RUN   TestPageCompressAlgoMixedWithScan
--- PASS: TestPageCompressAlgoMixedWithScan (6.29s)
=== RUN   TestWrittenDataSz
--- PASS: TestWrittenDataSz (4.23s)
=== RUN   TestWrittenDataSzAfterRecoveryCleaning
--- PASS: TestWrittenDataSzAfterRecoveryCleaning (4.62s)
=== RUN   TestWrittenHdrSz
--- PASS: TestWrittenHdrSz (4.02s)
=== RUN   TestPersistConfigUpgrade
--- PASS: TestPersistConfigUpgrade (0.01s)
=== RUN   TestLSSSegmentSize
--- PASS: TestLSSSegmentSize (0.24s)
=== RUN   TestPlasmaFlushBufferSzCfg
--- PASS: TestPlasmaFlushBufferSzCfg (0.16s)
=== RUN   TestCompactionCountwithCompactFullMarshal
--- PASS: TestCompactionCountwithCompactFullMarshal (0.09s)
=== RUN   TestCompactionCountwithCompactFullMarshalSMO
--- PASS: TestCompactionCountwithCompactFullMarshalSMO (0.05s)
=== RUN   TestPageHasDataRecordsOnCompactFullMarshal
--- PASS: TestPageHasDataRecordsOnCompactFullMarshal (0.09s)
=== RUN   TestPauseReaderOnLowMemory
--- PASS: TestPauseReaderOnLowMemory (1.05s)
=== RUN   TestRecoveryCleanerFragRatio
--- PASS: TestRecoveryCleanerFragRatio (224.60s)
=== RUN   TestRecoveryCleanerRelocation
--- PASS: TestRecoveryCleanerRelocation (224.75s)
=== RUN   TestRecoveryCleanerDataSize
--- PASS: TestRecoveryCleanerDataSize (224.74s)
=== RUN   TestRecoveryCleanerDeleteInstance
--- PASS: TestRecoveryCleanerDeleteInstance (439.94s)
=== RUN   TestRecoveryCleanerRecoveryPoint
--- PASS: TestRecoveryCleanerRecoveryPoint (41.89s)
=== RUN   TestRecoveryCleanerCorruptInstance
--- PASS: TestRecoveryCleanerCorruptInstance (0.21s)
=== RUN   TestRecoveryCleanerAhead
--- PASS: TestRecoveryCleanerAhead (4.25s)
=== RUN   TestRecoveryCleanerAheadAfterRecovery
--- PASS: TestRecoveryCleanerAheadAfterRecovery (2.24s)
=== RUN   TestCleaningUncommittedData
--- PASS: TestCleaningUncommittedData (0.04s)
=== RUN   TestPlasmaRecoverySimple
--- PASS: TestPlasmaRecoverySimple (0.06s)
=== RUN   TestPlasmaRecovery
--- PASS: TestPlasmaRecovery (27.47s)
=== RUN   TestRecoverDedicatedAsShared
--- PASS: TestRecoverDedicatedAsShared (0.12s)
=== RUN   TestRecoverDedicatedAsSharedDeleteShardMeta
--- PASS: TestRecoverDedicatedAsSharedDeleteShardMeta (0.12s)
=== RUN   TestRecoverDedicatedAsSharedCorruptDataLog
--- PASS: TestRecoverDedicatedAsSharedCorruptDataLog (0.08s)
=== RUN   TestShardRecoveryShared
--- PASS: TestShardRecoveryShared (15.73s)
=== RUN   TestShardRecoveryRecoveryLogAhead
--- PASS: TestShardRecoveryRecoveryLogAhead (47.96s)
=== RUN   TestShardRecoveryDataLogAhead
--- PASS: TestShardRecoveryDataLogAhead (31.88s)
=== RUN   TestShardRecoveryDestroyBlksInDataLog
--- PASS: TestShardRecoveryDestroyBlksInDataLog (14.86s)
=== RUN   TestShardRecoveryDestroyBlksInRecoveryLog
--- PASS: TestShardRecoveryDestroyBlksInRecoveryLog (15.14s)
=== RUN   TestShardRecoveryDestroyBlksInBothLog
--- PASS: TestShardRecoveryDestroyBlksInBothLog (14.84s)
=== RUN   TestShardRecoveryRecoveryLogCorruption
--- PASS: TestShardRecoveryRecoveryLogCorruption (14.59s)
=== RUN   TestShardRecoveryRecoveryLogCorruptionServerless
--- PASS: TestShardRecoveryRecoveryLogCorruptionServerless (14.63s)
=== RUN   TestShardRecoveryDataLogCorruption
--- PASS: TestShardRecoveryDataLogCorruption (15.37s)
=== RUN   TestShardRecoveryDataLogCorruptionServerless
--- PASS: TestShardRecoveryDataLogCorruptionServerless (15.90s)
=== RUN   TestShardRecoverySharedNoRP
--- PASS: TestShardRecoverySharedNoRP (15.32s)
=== RUN   TestShardRecoveryNotEnoughMem
--- PASS: TestShardRecoveryNotEnoughMem (39.47s)
=== RUN   TestShardRecoveryCleanup
--- PASS: TestShardRecoveryCleanup (0.47s)
=== RUN   TestShardRecoveryRebuildSharedLog
--- PASS: TestShardRecoveryRebuildSharedLog (1.82s)
=== RUN   TestShardRecoveryUpgradeWithCheckpoint
--- PASS: TestShardRecoveryUpgradeWithCheckpoint (0.62s)
=== RUN   TestShardRecoveryUpgradeWithLogReplay
--- PASS: TestShardRecoveryUpgradeWithLogReplay (0.61s)
=== RUN   TestShardRecoveryRebuildAfterError
--- PASS: TestShardRecoveryRebuildAfterError (1.85s)
=== RUN   TestShardRecoveryRebuildAfterConcurrentDelete
--- PASS: TestShardRecoveryRebuildAfterConcurrentDelete (2.68s)
=== RUN   TestShardRecoveryAfterDeleteInstance
--- PASS: TestShardRecoveryAfterDeleteInstance (0.13s)
=== RUN   TestShardRecoveryDestroyShard
--- PASS: TestShardRecoveryDestroyShard (0.26s)
=== RUN   TestHeaderRepair
--- PASS: TestHeaderRepair (0.10s)
=== RUN   TestCheckpointWithWriter
--- PASS: TestCheckpointWithWriter (4.72s)
=== RUN   TestPlasmaRecoveryWithRepairFullReplay
--- PASS: TestPlasmaRecoveryWithRepairFullReplay (32.91s)
=== RUN   TestPlasmaRecoveryWithInsertRepairCheckpoint
--- PASS: TestPlasmaRecoveryWithInsertRepairCheckpoint (34.95s)
=== RUN   TestPlasmaRecoveryWithDeleteRepairCheckpoint
--- PASS: TestPlasmaRecoveryWithDeleteRepairCheckpoint (12.95s)
=== RUN   TestShardRecoverySharedFullReplayOnError
--- PASS: TestShardRecoverySharedFullReplayOnError (17.10s)
=== RUN   TestShardRecoverySharedFullReplayOnErrorServerless
--- PASS: TestShardRecoverySharedFullReplayOnErrorServerless (17.96s)
=== RUN   TestShardRecoveryDedicatedFullReplayOnError
--- PASS: TestShardRecoveryDedicatedFullReplayOnError (17.24s)
=== RUN   TestShardRecoveryDedicatedFullReplayOnErrorServerless
--- PASS: TestShardRecoveryDedicatedFullReplayOnErrorServerless (17.92s)
=== RUN   TestShardRecoverySharedFullReplayOnErrorWithRepair
--- PASS: TestShardRecoverySharedFullReplayOnErrorWithRepair (19.24s)
=== RUN   TestGlobalWorkContextForRecovery
--- PASS: TestGlobalWorkContextForRecovery (0.48s)
=== RUN   TestShardRecoveryPartialMetadata
--- PASS: TestShardRecoveryPartialMetadata (0.12s)
=== RUN   TestShardRecoveryPartialLSSMetadata
--- PASS: TestShardRecoveryPartialLSSMetadata (0.10s)
=== RUN   TestShardRecoveryPartialSkiplog
--- PASS: TestShardRecoveryPartialSkiplog (0.10s)
=== RUN   TestShardRecoveryTempSkiplog
--- PASS: TestShardRecoveryTempSkiplog (0.27s)
=== RUN   TestPageRemovalLargeKey
--- PASS: TestPageRemovalLargeKey (0.05s)
=== RUN   TestRpVersionOverflow
--- PASS: TestRpVersionOverflow (0.06s)
=== RUN   TestRecoveryCancelHeaderReplay
--- PASS: TestRecoveryCancelHeaderReplay (2.45s)
=== RUN   TestRecoveryCancelWithDelay
--- PASS: TestRecoveryCancelWithDelay (2.54s)
=== RUN   TestRecoveryCancelWithDataReplay
--- PASS: TestRecoveryCancelWithDataReplay (0.32s)
=== RUN   TestShardRecoveryInstPathErrorString
--- PASS: TestShardRecoveryInstPathErrorString (0.05s)
=== RUN   TestSkipLogSimple
--- PASS: TestSkipLogSimple (0.00s)
=== RUN   TestSkipLogLoadStore
--- PASS: TestSkipLogLoadStore (0.03s)
=== RUN   TestRPCSimple
--- PASS: TestRPCSimple (0.02s)
=== RUN   TestRPCOpenClose
--- PASS: TestRPCOpenClose (4.05s)
=== RUN   TestRPCWritePartialPart
--- PASS: TestRPCWritePartialPart (0.03s)
=== RUN   TestRPCWriteMultiPart
--- PASS: TestRPCWriteMultiPart (15.32s)
=== RUN   TestRPCWriteUnAlignedHeadTailWithChecksum
--- PASS: TestRPCWriteUnAlignedHeadTailWithChecksum (11.22s)
=== RUN   TestRPCDoubleOpenWithStaleWrites
--- PASS: TestRPCDoubleOpenWithStaleWrites (22.16s)
=== RUN   TestRPCRmDirBusy
--- PASS: TestRPCRmDirBusy (4.66s)
=== RUN   TestRPCServerGCIdleHandle
--- PASS: TestRPCServerGCIdleHandle (15.01s)
=== RUN   TestRPCServerShutdownWithIO
--- PASS: TestRPCServerShutdownWithIO (11.37s)
=== RUN   TestRPCServerShutdownBusy
--- PASS: TestRPCServerShutdownBusy (10.07s)
=== RUN   TestRPCServerBindWithRetry
--- PASS: TestRPCServerBindWithRetry (1.50s)
=== RUN   TestRPCClientWithCorruptRequest
--- PASS: TestRPCClientWithCorruptRequest (3.14s)
=== RUN   TestRPCClientWithCorruptReqHeaders
--- PASS: TestRPCClientWithCorruptReqHeaders (0.02s)
=== RUN   TestRPCWithCorruptResponse
--- PASS: TestRPCWithCorruptResponse (0.01s)
=== RUN   TestRPCVersion
--- PASS: TestRPCVersion (0.00s)
=== RUN   TestRPCSyncFileIdempotent
--- PASS: TestRPCSyncFileIdempotent (10.06s)
=== RUN   TestRPCSyncFileError
--- PASS: TestRPCSyncFileError (0.01s)
=== RUN   TestRPCOpenWithRetry
--- PASS: TestRPCOpenWithRetry (5.01s)
=== RUN   TestRPCOpenWithDialTimeout
--- PASS: TestRPCOpenWithDialTimeout (0.02s)
=== RUN   TestRPCWriteRetryWithDigest
--- PASS: TestRPCWriteRetryWithDigest (40.21s)
=== RUN   TestRPCWithThrottle
--- PASS: TestRPCWithThrottle (37.38s)
=== RUN   TestRPCRateControl
--- PASS: TestRPCRateControl (190.94s)
=== RUN   TestRPCRateControlII
--- PASS: TestRPCRateControlII (17.23s)
=== RUN   TestRPCWriteEmptyFile
--- PASS: TestRPCWriteEmptyFile (0.02s)
=== RUN   TestRPCPersistentConnections
--- PASS: TestRPCPersistentConnections (2.00s)
=== RUN   TestRPCClientEOF
--- PASS: TestRPCClientEOF (9.32s)
=== RUN   TestRPCServerWithHTTPS
--- PASS: TestRPCServerWithHTTPS (9.80s)
=== RUN   TestRPCServerQuota
--- PASS: TestRPCServerQuota (11.07s)
=== RUN   TestRPCServerChannelConfig
--- PASS: TestRPCServerChannelConfig (0.17s)
=== RUN   TestShardMetadata
--- PASS: TestShardMetadata (0.85s)
=== RUN   TestPlasmaId
--- PASS: TestPlasmaId (0.06s)
=== RUN   TestShardPersistence
--- PASS: TestShardPersistence (0.31s)
=== RUN   TestShardDestroy
--- PASS: TestShardDestroy (0.04s)
=== RUN   TestShardClose
--- PASS: TestShardClose (5.04s)
=== RUN   TestShardMgrRecovery
--- PASS: TestShardMgrRecovery (0.09s)
=== RUN   TestShardDeadData
--- PASS: TestShardDeadData (0.31s)
=== RUN   TestShardDeadDataNegativeFlushDataSize
--- PASS: TestShardDeadDataNegativeFlushDataSize (0.53s)
=== RUN   TestShardConfigUpdate
--- PASS: TestShardConfigUpdate (0.05s)
=== RUN   TestShardWriteAmp
--- PASS: TestShardWriteAmp (10.16s)
=== RUN   TestShardStats
--- PASS: TestShardStats (0.26s)
=== RUN   TestShardCompressionStats
--- PASS: TestShardCompressionStats (2.38s)
=== RUN   TestShardMultipleWriters
--- PASS: TestShardMultipleWriters (0.19s)
=== RUN   TestShardDestroyMultiple
--- PASS: TestShardDestroyMultiple (0.14s)
=== RUN   TestShardBackupCorrupted
--- PASS: TestShardBackupCorrupted (0.14s)
=== RUN   TestShardBackupCorruptedShare
--- PASS: TestShardBackupCorruptedShare (0.06s)
=== RUN   TestShardCorruption
--- PASS: TestShardCorruption (0.14s)
=== RUN   TestShardCorruptionAddInstance
--- PASS: TestShardCorruptionAddInstance (0.21s)
=== RUN   TestShardCreateError
--- PASS: TestShardCreateError (0.24s)
=== RUN   TestShardNumInsts
--- PASS: TestShardNumInsts (1.66s)
=== RUN   TestShardInstanceGroup
--- PASS: TestShardInstanceGroup (0.07s)
=== RUN   TestShardLeak
--- PASS: TestShardLeak (2.05s)
=== RUN   TestShardMemLeak
--- PASS: TestShardMemLeak (1.08s)
=== RUN   TestShardFind
--- PASS: TestShardFind (0.23s)
=== RUN   TestShardFileOpenDescCount
--- PASS: TestShardFileOpenDescCount (87.84s)
=== RUN   TestShardShutdownSharedLSS
--- PASS: TestShardShutdownSharedLSS (11.69s)
=== RUN   TestShardUUIDChange
--- PASS: TestShardUUIDChange (0.06s)
=== RUN   TestShardUUIDStable
--- PASS: TestShardUUIDStable (0.04s)
=== RUN   TestShardUUIDUpgradeDedicated
--- PASS: TestShardUUIDUpgradeDedicated (0.10s)
=== RUN   TestShardUUIDUpgradeShared
--- PASS: TestShardUUIDUpgradeShared (0.08s)
=== RUN   TestShardMetadataPersistenceOnPanic
--- PASS: TestShardMetadataPersistenceOnPanic (0.17s)
=== RUN   TestShardAlternateId
--- PASS: TestShardAlternateId (0.14s)
=== RUN   TestSMRSimple
--- PASS: TestSMRSimple (1.09s)
=== RUN   TestSMRConcurrent
--- PASS: TestSMRConcurrent (73.27s)
=== RUN   TestSMRComplex
--- PASS: TestSMRComplex (149.57s)
=== RUN   TestDGMWithCASConflicts
--- PASS: TestDGMWithCASConflicts (41.41s)
=== RUN   TestMaxSMRPendingMem
--- PASS: TestMaxSMRPendingMem (0.01s)
=== RUN   TestStatsLogger
--- PASS: TestStatsLogger (20.44s)
=== RUN   TestStatsSamplePercentile
--- PASS: TestStatsSamplePercentile (0.01s)
=== RUN   TestPlasmaSwapper
--- PASS: TestPlasmaSwapper (21.15s)
=== RUN   TestPlasmaAutoSwapper
--- PASS: TestPlasmaAutoSwapper (85.78s)
=== RUN   TestSwapperAddInstance
--- PASS: TestSwapperAddInstance (4.60s)
=== RUN   TestSwapperRemoveInstance
--- PASS: TestSwapperRemoveInstance (4.43s)
=== RUN   TestSwapperJoinContext
--- PASS: TestSwapperJoinContext (4.94s)
=== RUN   TestSwapperSplitContext
--- PASS: TestSwapperSplitContext (4.91s)
=== RUN   TestSwapperGlobalClock
--- PASS: TestSwapperGlobalClock (29.93s)
=== RUN   TestSwapperConflict
--- PASS: TestSwapperConflict (2.88s)
=== RUN   TestSwapperRemoveInstanceWait
--- PASS: TestSwapperRemoveInstanceWait (3.43s)
=== RUN   TestSwapperStats
--- PASS: TestSwapperStats (1.03s)
=== RUN   TestSwapperSweepInterval
--- PASS: TestSwapperSweepInterval (0.47s)
=== RUN   TestSweepCompress
--- PASS: TestSweepCompress (0.07s)
=== RUN   TestTenantShardAssignment
--- PASS: TestTenantShardAssignment (4.14s)
=== RUN   TestTenantShardAssignmentServerless
--- PASS: TestTenantShardAssignmentServerless (4.14s)
=== RUN   TestTenantShardAssignmentOnPrem
--- PASS: TestTenantShardAssignmentOnPrem (4.23s)
=== RUN   TestTenantShardAssignmentDedicated
--- PASS: TestTenantShardAssignmentDedicated (2.19s)
=== RUN   TestTenantShardAssignmentDedicatedMainBackIndexes
--- PASS: TestTenantShardAssignmentDedicatedMainBackIndexes (0.09s)
=== RUN   TestTenantShardRecovery
--- PASS: TestTenantShardRecovery (3.99s)
=== RUN   TestTenantMemUsed
--- PASS: TestTenantMemUsed (3.82s)
=== RUN   TestTenantSwitchController
--- PASS: TestTenantSwitchController (0.11s)
=== RUN   TestTenantSwitchControllerActiveWCtx
--- PASS: TestTenantSwitchControllerActiveWCtx (0.27s)
=== RUN   TestTenantSwitchControllerDuringFlush
--- PASS: TestTenantSwitchControllerDuringFlush (4.04s)
=== RUN   TestTenantBuildController
--- PASS: TestTenantBuildController (2.09s)
=== RUN   TestTenantBuildControllerOnPrem
--- PASS: TestTenantBuildControllerOnPrem (2.12s)
=== RUN   TestTenantSingleController
--- PASS: TestTenantSingleController (2.17s)
=== RUN   TestTenantControllerSwapperIncremental
--- PASS: TestTenantControllerSwapperIncremental (1.37s)
=== RUN   TestTenantControllerSwapperInitial
--- PASS: TestTenantControllerSwapperInitial (1.35s)
=== RUN   TestTenantControllerSwapperPeriodic
--- PASS: TestTenantControllerSwapperPeriodic (3.52s)
=== RUN   TestTenantSwapperZeroResident
--- PASS: TestTenantSwapperZeroResident (2.76s)
=== RUN   TestTenantSwapperZeroItem
--- PASS: TestTenantSwapperZeroItem (1.14s)
=== RUN   TestTenantAssignMandatoryQuota
--- PASS: TestTenantAssignMandatoryQuota (0.63s)
=== RUN   TestTenantMutationQuota
--- PASS: TestTenantMutationQuota (0.16s)
=== RUN   TestTenantMutationQuotaInitial
--- PASS: TestTenantMutationQuotaInitial (0.16s)
=== RUN   TestTenantMutationQuotaEmpty
--- PASS: TestTenantMutationQuotaEmpty (0.08s)
=== RUN   TestTenantMutationQuotaIncremental
--- PASS: TestTenantMutationQuotaIncremental (0.14s)
=== RUN   TestTenantCalibrateMutationQuota
--- PASS: TestTenantCalibrateMutationQuota (0.06s)
=== RUN   TestTenantMutationRate
--- PASS: TestTenantMutationRate (0.32s)
=== RUN   TestTenantMutationQuotaAdjustRate
--- PASS: TestTenantMutationQuotaAdjustRate (0.46s)
=== RUN   TestTenantBuildQuota
--- PASS: TestTenantBuildQuota (0.07s)
=== RUN   TestTenantInitialBuildQuota
--- PASS: TestTenantInitialBuildQuota (0.06s)
=== RUN   TestTenantInitialBuildNonDGM
--- PASS: TestTenantInitialBuildNonDGM (3.16s)
=== RUN   TestTenantInitialBuildDGM
--- PASS: TestTenantInitialBuildDGM (3.17s)
=== RUN   TestTenantInitialBuildZeroResident
--- PASS: TestTenantInitialBuildZeroResident (3.24s)
=== RUN   TestTenantIncrementalBuildDGM
--- PASS: TestTenantIncrementalBuildDGM (5.08s)
=== RUN   TestTenantInitialBuildTwoTenants
--- PASS: TestTenantInitialBuildTwoTenants (5.06s)
=== RUN   TestTenantInitialBuildTwoControllers
--- PASS: TestTenantInitialBuildTwoControllers (5.10s)
=== RUN   TestTenantIncrementalBuildTwoIndexes
--- PASS: TestTenantIncrementalBuildTwoIndexes (0.57s)
=== RUN   TestTenantIncrementalBuildConcurrent
--- PASS: TestTenantIncrementalBuildConcurrent (7.58s)
=== RUN   TestTenantMixedConcurrentSingleContainer
--- PASS: TestTenantMixedConcurrentSingleContainer (12.52s)
=== RUN   TestTenantDecrementGlobalQuota
--- PASS: TestTenantDecrementGlobalQuota (3.78s)
=== RUN   TestTenantInitialBuildNotEnoughQuota
--- PASS: TestTenantInitialBuildNotEnoughQuota (5.04s)
=== RUN   TestTenantRecoveryResidentRatioHeaderReplay
--- PASS: TestTenantRecoveryResidentRatioHeaderReplay (0.21s)
=== RUN   TestTenantRecoveryResidentRatioDataReplay
--- PASS: TestTenantRecoveryResidentRatioDataReplay (0.23s)
=== RUN   TestTenantRecoveryController
--- PASS: TestTenantRecoveryController (2.56s)
=== RUN   TestTenantRecoveryControllerOnPrem
--- PASS: TestTenantRecoveryControllerOnPrem (2.59s)
=== RUN   TestTenantRecoveryQuotaWithLastCheckpoint
--- PASS: TestTenantRecoveryQuotaWithLastCheckpoint (1.28s)
=== RUN   TestTenantRecoveryQuotaZeroResidentWithLastCheckpoint
--- PASS: TestTenantRecoveryQuotaZeroResidentWithLastCheckpoint (5.11s)
=== RUN   TestTenantRecoveryQuotaWithFormula
--- PASS: TestTenantRecoveryQuotaWithFormula (5.05s)
=== RUN   TestTenantRecoveryQuotaWithDataReplay
--- PASS: TestTenantRecoveryQuotaWithDataReplay (10.96s)
=== RUN   TestTenantRecoveryEvictionNoCheckpoint
--- PASS: TestTenantRecoveryEvictionNoCheckpoint (24.85s)
=== RUN   TestTenantRecoveryEvictionHeaderReplay
--- PASS: TestTenantRecoveryEvictionHeaderReplay (14.23s)
=== RUN   TestTenantRecoveryEvictionDataReplaySequential
--- PASS: TestTenantRecoveryEvictionDataReplaySequential (13.77s)
=== RUN   TestTenantRecoveryEvictionDataReplayInterleaved
--- PASS: TestTenantRecoveryEvictionDataReplayInterleaved (15.52s)
=== RUN   TestTenantRecoveryEvictionDataReplayInterleavedSingleContainer
--- PASS: TestTenantRecoveryEvictionDataReplayInterleavedSingleContainer (9.34s)
=== RUN   TestTenantRecoveryEvictionDataReplayInterleavedSingleContainerTwoTenants
--- PASS: TestTenantRecoveryEvictionDataReplayInterleavedSingleContainerTwoTenants (10.30s)
=== RUN   TestTenantRecoveryEvictionDataReplayNoCheckpoint
--- PASS: TestTenantRecoveryEvictionDataReplayNoCheckpoint (15.51s)
=== RUN   TestTenantRecoveryEvictionDataReplaySingle
--- PASS: TestTenantRecoveryEvictionDataReplaySingle (7.09s)
=== RUN   TestTenantRecoveryLastCheckpoint
--- PASS: TestTenantRecoveryLastCheckpoint (8.26s)
=== RUN   TestTenantRecoveryRequestQuota
--- PASS: TestTenantRecoveryRequestQuota (4.08s)
=== RUN   TestTenantWorkingSetPageHits
--- PASS: TestTenantWorkingSetPageHits (0.02s)
=== RUN   TestTenantWorkingSetConcurrent
--- PASS: TestTenantWorkingSetConcurrent (0.25s)
=== RUN   TestTenantDiscretionaryQuotaNoScan
--- PASS: TestTenantDiscretionaryQuotaNoScan (3.26s)
=== RUN   TestTenantDiscretionaryQuotaScan
--- PASS: TestTenantDiscretionaryQuotaScan (12.37s)
=== RUN   TestTenantAssignDiscretionaryQuota
--- PASS: TestTenantAssignDiscretionaryQuota (36.84s)
=== RUN   TestTenantMinimumQuota
--- PASS: TestTenantMinimumQuota (12.55s)
=== RUN   TestTenantWorkingSetIdle
--- PASS: TestTenantWorkingSetIdle (0.02s)
=== RUN   TestTenantWorkingSetSinceIdle
--- PASS: TestTenantWorkingSetSinceIdle (14.44s)
=== RUN   TestTenantWorkingSetIdleTime
--- PASS: TestTenantWorkingSetIdleTime (1.02s)
=== RUN   TestTenantAssignIdleQuota
--- PASS: TestTenantAssignIdleQuota (12.48s)
=== RUN   TestTenantAssignIdleQuotaEmptyIndex
--- PASS: TestTenantAssignIdleQuotaEmptyIndex (0.17s)
=== RUN   TestTenantActivateIdleTenant
--- PASS: TestTenantActivateIdleTenant (37.15s)
=== RUN   TestTenantActivateIdleTenantWriter
--- PASS: TestTenantActivateIdleTenantWriter (36.81s)
=== RUN   TestTenantActivateIdleTenantZeroQuota
--- PASS: TestTenantActivateIdleTenantZeroQuota (0.78s)
=== RUN   TestTenantStatsInStr
--- PASS: TestTenantStatsInStr (0.02s)
=== RUN   TestTenantRuntimeStats
--- PASS: TestTenantRuntimeStats (0.19s)
=== RUN   TestTenantMandatoryQuotaLowResident
--- PASS: TestTenantMandatoryQuotaLowResident (61.18s)
=== RUN   TestTenantMandatoryDeleteIndex
--- PASS: TestTenantMandatoryDeleteIndex (64.34s)
=== RUN   TestTenantMandatoryQuotaSmallIndex
--- PASS: TestTenantMandatoryQuotaSmallIndex (0.06s)
=== RUN   TestTenantNumShards
--- PASS: TestTenantNumShards (1.86s)
=== RUN   TestTenantNumShardsMixedConfig
--- PASS: TestTenantNumShardsMixedConfig (3.02s)
=== RUN   TestMVCCPurgeIdleTenant
--- PASS: TestMVCCPurgeIdleTenant (16.37s)
=== RUN   TestTenantMemQuotaAfterRecoveryOnPrem
--- PASS: TestTenantMemQuotaAfterRecoveryOnPrem (1.54s)
=== RUN   TestTenantShardLSSSegmentSizeOnPremVsServerless
--- PASS: TestTenantShardLSSSegmentSizeOnPremVsServerless (7.94s)
=== RUN   TestWriteRenameFileWithSync
--- PASS: TestWriteRenameFileWithSync (0.01s)
=== RUN   TestHistogram
--- PASS: TestHistogram (0.00s)
=== RUN   TestSyncFilePanicUnstable
--- PASS: TestSyncFilePanicUnstable (0.00s)
=== RUN   TestSyncFileNoPanicUnstable
--- PASS: TestSyncFileNoPanicUnstable (0.00s)
=== RUN   TestSCtx
--- PASS: TestSCtx (17.44s)
=== RUN   TestWCtxGeneric
--- PASS: TestWCtxGeneric (21.13s)
=== RUN   TestWCtxWriter
--- PASS: TestWCtxWriter (21.95s)
=== RUN   TestSCtxTrimWithReader
--- PASS: TestSCtxTrimWithReader (0.03s)
=== RUN   TestSCtxTrimWithWriter
--- PASS: TestSCtxTrimWithWriter (0.03s)
=== RUN   TestSCtxTrimEmpty
--- PASS: TestSCtxTrimEmpty (0.01s)
=== RUN   TestWCtxTrimWithReader
--- PASS: TestWCtxTrimWithReader (0.03s)
=== RUN   TestWCtxTrimWithWriter
--- PASS: TestWCtxTrimWithWriter (0.03s)
--- PASS: TestExtrasN1 (0.00s)
--- PASS: TestExtrasN3 (0.00s)
--- PASS: TestExtrasN2 (0.00s)
PASS
ok  	github.com/couchbase/plasma	7340.226s
=== RUN   TestInteger
--- PASS: TestInteger (0.00s)
=== RUN   TestSmallDecimal
--- PASS: TestSmallDecimal (0.00s)
=== RUN   TestLargeDecimal
--- PASS: TestLargeDecimal (0.00s)
=== RUN   TestFloat
--- PASS: TestFloat (0.00s)
=== RUN   TestSuffixCoding
--- PASS: TestSuffixCoding (0.00s)
=== RUN   TestCodecLength
--- PASS: TestCodecLength (0.00s)
=== RUN   TestSpecialString
--- PASS: TestSpecialString (0.00s)
=== RUN   TestCodecNoLength
--- PASS: TestCodecNoLength (0.00s)
=== RUN   TestCodecJSON
--- PASS: TestCodecJSON (0.00s)
=== RUN   TestReference
--- PASS: TestReference (0.00s)
=== RUN   TestN1QLEncode
--- PASS: TestN1QLEncode (0.00s)
=== RUN   TestArrayExplodeJoin
--- PASS: TestArrayExplodeJoin (0.00s)
=== RUN   TestN1QLDecode
--- PASS: TestN1QLDecode (0.00s)
=== RUN   TestN1QLDecode2
--- PASS: TestN1QLDecode2 (0.00s)
=== RUN   TestArrayExplodeJoin2
--- PASS: TestArrayExplodeJoin2 (0.00s)
=== RUN   TestMB28956
--- PASS: TestMB28956 (0.00s)
=== RUN   TestFixEncodedInt
--- PASS: TestFixEncodedInt (0.00s)
=== RUN   TestN1QLDecodeLargeInt64
--- PASS: TestN1QLDecodeLargeInt64 (0.00s)
=== RUN   TestMixedModeFixEncodedInt
TESTING [4111686018427387900, -8223372036854775808, 822337203685477618] 
PASS 
TESTING [0] 
PASS 
TESTING [0.0] 
PASS 
TESTING [0.0000] 
PASS 
TESTING [0.0000000] 
PASS 
TESTING [-0] 
PASS 
TESTING [-0.0] 
PASS 
TESTING [-0.0000] 
PASS 
TESTING [-0.0000000] 
PASS 
TESTING [1] 
PASS 
TESTING [20] 
PASS 
TESTING [3456] 
PASS 
TESTING [7645000] 
PASS 
TESTING [9223372036854775807] 
PASS 
TESTING [9223372036854775806] 
PASS 
TESTING [9223372036854775808] 
PASS 
TESTING [92233720368547758071234000] 
PASS 
TESTING [92233720368547758071234987437653] 
PASS 
TESTING [12300000000000000000000000000000056] 
PASS 
TESTING [12300000000000000000000000000000000] 
PASS 
TESTING [123000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000056] 
PASS 
TESTING [123000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000] 
PASS 
TESTING [12300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000056] 
PASS 
TESTING [210690] 
PASS 
TESTING [90000] 
PASS 
TESTING [123000000] 
PASS 
TESTING [3.60e2] 
PASS 
TESTING [36e2] 
PASS 
TESTING [1.9999999999e10] 
PASS 
TESTING [1.99999e10] 
PASS 
TESTING [1.99999e5] 
PASS 
TESTING [0.00000000000012e15] 
PASS 
TESTING [7.64507352e8] 
PASS 
TESTING [9.2233720368547758071234987437653e31] 
PASS 
TESTING [2650e-1] 
PASS 
TESTING [26500e-1] 
PASS 
TESTING [-1] 
PASS 
TESTING [-20] 
PASS 
TESTING [-3456] 
PASS 
TESTING [-7645000] 
PASS 
TESTING [-9223372036854775808] 
PASS 
TESTING [-9223372036854775807] 
PASS 
TESTING [-9223372036854775806] 
PASS 
TESTING [-9223372036854775809] 
PASS 
TESTING [-92233720368547758071234000] 
PASS 
TESTING [-92233720368547758071234987437653] 
PASS 
TESTING [-12300000000000000000000000000000056] 
PASS 
TESTING [-12300000000000000000000000000000000] 
PASS 
TESTING [-123000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000056] 
PASS 
TESTING [-123000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000] 
PASS 
TESTING [-210690] 
PASS 
TESTING [-90000] 
PASS 
TESTING [-123000000] 
PASS 
TESTING [-3.60e2] 
PASS 
TESTING [-36e2] 
PASS 
TESTING [-1.9999999999e10] 
PASS 
TESTING [-1.99999e10] 
PASS 
TESTING [-1.99999e5] 
PASS 
TESTING [-0.00000000000012e15] 
PASS 
TESTING [-2650e-1] 
PASS 
TESTING [-26500e-1] 
PASS 
TESTING [0.03] 
PASS 
TESTING [198.60] 
PASS 
TESTING [2000045.178] 
PASS 
TESTING [1.7976931348623157e+308] 
PASS 
TESTING [0.000000000000000000890] 
PASS 
TESTING [257953786.9864236576] 
PASS 
TESTING [257953786.9864236576e8] 
PASS 
TESTING [36.912e3] 
PASS 
TESTING [2761.67e0] 
PASS 
TESTING [2761.67e00] 
PASS 
TESTING [2761.67e000] 
PASS 
TESTING [7676546.67e-3] 
PASS 
TESTING [-0.03] 
PASS 
TESTING [-198.60] 
PASS 
TESTING [-2000045.178] 
PASS 
TESTING [-1.7976931348623157e+308] 
PASS 
TESTING [-0.000000000000000000890] 
PASS 
TESTING [-257953786.9864236576] 
PASS 
TESTING [-257953786.9864236576e8] 
PASS 
TESTING [-36.912e3] 
PASS 
TESTING [-2761.67e0] 
PASS 
TESTING [-2761.67e00] 
PASS 
TESTING [-2761.67e000] 
PASS 
TESTING [-7676546.67e-3] 
PASS 
--- PASS: TestMixedModeFixEncodedInt (0.01s)
=== RUN   TestCodecDesc
--- PASS: TestCodecDesc (0.00s)
=== RUN   TestCodecDescPropLen
--- PASS: TestCodecDescPropLen (0.00s)
=== RUN   TestCodecDescSplChar
--- PASS: TestCodecDescSplChar (0.00s)
PASS
ok  	github.com/couchbase/indexing/secondary/collatejson	0.039s
=== RUN   TestForestDBIterator
2023-09-01T13:09:14.609+05:30 [INFO][FDB] Forestdb blockcache size 134217728 initialized in 5552 us

2023-09-01T13:09:14.610+05:30 [INFO][FDB] Forestdb opened database file test
2023-09-01T13:09:14.613+05:30 [INFO][FDB] Forestdb closed database file test
--- PASS: TestForestDBIterator (0.01s)
=== RUN   TestForestDBIteratorSeek
2023-09-01T13:09:14.614+05:30 [INFO][FDB] Forestdb opened database file test
2023-09-01T13:09:14.619+05:30 [INFO][FDB] Forestdb closed database file test
--- PASS: TestForestDBIteratorSeek (0.01s)
=== RUN   TestPrimaryIndexEntry
--- PASS: TestPrimaryIndexEntry (0.00s)
=== RUN   TestSecondaryIndexEntry
--- PASS: TestSecondaryIndexEntry (0.00s)
=== RUN   TestPrimaryIndexEntryMatch
--- PASS: TestPrimaryIndexEntryMatch (0.00s)
=== RUN   TestSecondaryIndexEntryMatch
--- PASS: TestSecondaryIndexEntryMatch (0.00s)
=== RUN   TestLongDocIdEntry
--- PASS: TestLongDocIdEntry (0.00s)
=== RUN   TestMemDBInsertionPerf
Maximum number of file descriptors = 200000
Set IO Concurrency: 7200
2023-09-01T13:09:15.697+05:30 DEBU REGU.impl.init.0.func1() at aggrecorder.go:57 [id 42] will report aggregate recorder stats every 5m0s 
Initial build: 10000000 items took 1m48.578706136s -> 92099.08973748982 items/s
Incr build: 10000000 items took 2m4.853542312s -> 80093.84287240102 items/s
Main Index: {
"node_count":             18000000,
"soft_deletes":           0,
"read_conflicts":         0,
"insert_conflicts":       3,
"next_pointers_per_node": 1.3335,
"memory_used":            1695949116,
"node_allocs":            18000000,
"node_frees":             0,
"level_node_distribution":{
"level0": 13500101,
"level1": 3372532,
"level2": 845858,
"level3": 210944,
"level4": 52850,
"level5": 13336,
"level6": 3284,
"level7": 810,
"level8": 219,
"level9": 51,
"level10": 14,
"level11": 1,
"level12": 0,
"level13": 0,
"level14": 0,
"level15": 0,
"level16": 0,
"level17": 0,
"level18": 0,
"level19": 0,
"level20": 0,
"level21": 0,
"level22": 0,
"level23": 0,
"level24": 0,
"level25": 0,
"level26": 0,
"level27": 0,
"level28": 0,
"level29": 0,
"level30": 0,
"level31": 0,
"level32": 0
}
}
Back Index 0 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 1 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 2 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 3 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 4 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 5 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 6 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 7 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 8 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 9 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 10 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 11 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 12 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 13 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 14 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
Back Index 15 : {
"FastHTCount":  625000,
"SlowHTCount":  0,
"Conflicts":   0,
"MemoryInUse": 26250000
}
--- PASS: TestMemDBInsertionPerf (233.44s)
=== RUN   TestBasicsA
--- PASS: TestBasicsA (0.00s)
=== RUN   TestSizeA
--- PASS: TestSizeA (0.01s)
=== RUN   TestSizeWithFreelistA
--- PASS: TestSizeWithFreelistA (0.00s)
=== RUN   TestDequeueUptoSeqnoA
--- PASS: TestDequeueUptoSeqnoA (0.10s)
=== RUN   TestDequeueA
--- PASS: TestDequeueA (1.21s)
=== RUN   TestMultipleVbucketsA
--- PASS: TestMultipleVbucketsA (0.00s)
=== RUN   TestDequeueUptoFreelistA
--- PASS: TestDequeueUptoFreelistA (0.00s)
=== RUN   TestDequeueUptoFreelistMultVbA
--- PASS: TestDequeueUptoFreelistMultVbA (0.00s)
=== RUN   TestConcurrentEnqueueDequeueA
--- PASS: TestConcurrentEnqueueDequeueA (0.00s)
=== RUN   TestConcurrentEnqueueDequeueA1
--- PASS: TestConcurrentEnqueueDequeueA1 (10.01s)
=== RUN   TestEnqueueAppCh
--- PASS: TestEnqueueAppCh (2.00s)
=== RUN   TestDequeueN
--- PASS: TestDequeueN (0.00s)
=== RUN   TestConcurrentEnqueueDequeueN
--- PASS: TestConcurrentEnqueueDequeueN (0.00s)
=== RUN   TestConcurrentEnqueueDequeueN1
--- PASS: TestConcurrentEnqueueDequeueN1 (10.01s)
PASS
ok  	github.com/couchbase/indexing/secondary/indexer	257.525s
=== RUN   TestConnPoolBasicSanity
2023-09-01T13:13:39.246+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] started poolsize 3 overflow 6 low WM 3 relConn batch size 1 ...
2023-09-01T13:13:39.454+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] ... stopped
2023-09-01T13:13:40.247+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] Stopping releaseConnsRoutine
--- PASS: TestConnPoolBasicSanity (5.00s)
=== RUN   TestConnRelease
2023-09-01T13:13:44.250+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] started poolsize 500 overflow 10 low WM 40 relConn batch size 10 ...
Waiting for connections to get released
Waiting for more connections to get released
Waiting for further more connections to get released
2023-09-01T13:14:24.023+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] ... stopped
2023-09-01T13:14:24.268+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] Stopping releaseConnsRoutine
--- PASS: TestConnRelease (43.78s)
=== RUN   TestLongevity
2023-09-01T13:14:28.026+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] started poolsize 500 overflow 10 low WM 40 relConn batch size 10 ...
Releasing 4 conns.
Getting 3 conns.
Releasing 2 conns.
Getting 1 conns.
Releasing 2 conns.
Getting 1 conns.
Releasing 4 conns.
Getting 3 conns.
Releasing 2 conns.
Getting 1 conns.
Releasing 0 conns.
Getting 2 conns.
Releasing 3 conns.
Getting 4 conns.
Releasing 3 conns.
Getting 3 conns.
Releasing 2 conns.
Getting 2 conns.
Releasing 4 conns.
Getting 2 conns.
Releasing 1 conns.
Getting 1 conns.
Releasing 4 conns.
Getting 4 conns.
Releasing 4 conns.
Getting 1 conns.
Releasing 3 conns.
Getting 4 conns.
Releasing 4 conns.
Getting 2 conns.
2023-09-01T13:15:06.333+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] ... stopped
2023-09-01T13:15:07.045+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] Stopping releaseConnsRoutine
--- PASS: TestLongevity (42.31s)
=== RUN   TestSustainedHighConns
2023-09-01T13:15:10.334+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] started poolsize 500 overflow 10 low WM 40 relConn batch size 10 ...
Allocating 22 Connections
cp.curActConns = 0
Returning 1 Connections
cp.curActConns = 11
Returning 2 Connections
cp.curActConns = 19
Returning 4 Connections
Allocating 3 Connections
cp.curActConns = 18
Returning 1 Connections
Allocating 17 Connections
cp.curActConns = 22
Returning 3 Connections
cp.curActConns = 31
Returning 2 Connections
Allocating 23 Connections
cp.curActConns = 33
Returning 1 Connections
cp.curActConns = 44
Returning 4 Connections
cp.curActConns = 47
Returning 3 Connections
Allocating 12 Connections
cp.curActConns = 55
Returning 3 Connections
cp.curActConns = 53
Allocating 15 Connections
Returning 4 Connections
cp.curActConns = 64
Returning 0 Connections
Allocating 11 Connections
cp.curActConns = 66
Returning 3 Connections
cp.curActConns = 72
Returning 1 Connections
Allocating 2 Connections
cp.curActConns = 73
Returning 2 Connections
Allocating 18 Connections
cp.curActConns = 77
Returning 4 Connections
cp.curActConns = 85
Returning 3 Connections
Allocating 17 Connections
cp.curActConns = 88
Returning 4 Connections
cp.curActConns = 95
Returning 1 Connections
Allocating 8 Connections
cp.curActConns = 99
Returning 1 Connections
cp.curActConns = 101
Returning 1 Connections
Allocating 22 Connections
cp.curActConns = 110
Returning 4 Connections
cp.curActConns = 118
Returning 2 Connections
Allocating 20 Connections
cp.curActConns = 121
Returning 4 Connections
cp.curActConns = 132
Returning 3 Connections
Allocating 4 Connections
cp.curActConns = 133
Returning 0 Connections
Allocating 5 Connections
cp.curActConns = 133
Returning 3 Connections
cp.curActConns = 135
Returning 4 Connections
Allocating 10 Connections
cp.curActConns = 141
Returning 1 Connections
Allocating 5 Connections
cp.curActConns = 145
Returning 2 Connections
Allocating 20 Connections
cp.curActConns = 145
Returning 0 Connections
cp.curActConns = 157
Returning 2 Connections
cp.curActConns = 161
Returning 0 Connections
Allocating 12 Connections
cp.curActConns = 169
Returning 0 Connections
cp.curActConns = 173
Returning 0 Connections
Allocating 15 Connections
cp.curActConns = 181
Returning 1 Connections
cp.curActConns = 187
Returning 0 Connections
Allocating 6 Connections
cp.curActConns = 193
Returning 3 Connections
Allocating 23 Connections
cp.curActConns = 193
Returning 0 Connections
cp.curActConns = 205
Returning 1 Connections
cp.curActConns = 212
Returning 3 Connections
Allocating 3 Connections
cp.curActConns = 212
Returning 4 Connections
Allocating 4 Connections
cp.curActConns = 212
Returning 1 Connections
Allocating 1 Connections
cp.curActConns = 212
Returning 0 Connections
Allocating 3 Connections
cp.curActConns = 215
Returning 4 Connections
Allocating 4 Connections
cp.curActConns = 215
Returning 0 Connections
Allocating 2 Connections
cp.curActConns = 217
Returning 2 Connections
Allocating 1 Connections
cp.curActConns = 216
Returning 2 Connections
Allocating 3 Connections
cp.curActConns = 217
Returning 1 Connections
Allocating 0 Connections
cp.curActConns = 216
Returning 3 Connections
Allocating 2 Connections
cp.curActConns = 215
Returning 2 Connections
Allocating 0 Connections
cp.curActConns = 213
Returning 4 Connections
Allocating 1 Connections
cp.curActConns = 210
Returning 2 Connections
Allocating 2 Connections
cp.curActConns = 210
Returning 0 Connections
Allocating 3 Connections
cp.curActConns = 213
Returning 3 Connections
Allocating 0 Connections
cp.curActConns = 210
Returning 1 Connections
Allocating 0 Connections
cp.curActConns = 209
Returning 2 Connections
Allocating 2 Connections
cp.curActConns = 209
Returning 1 Connections
Allocating 4 Connections
cp.curActConns = 212
Returning 0 Connections
Allocating 3 Connections
cp.curActConns = 215
Returning 2 Connections
Allocating 2 Connections
cp.curActConns = 215
Returning 4 Connections
Allocating 1 Connections
cp.curActConns = 212
Returning 3 Connections
Allocating 0 Connections
cp.curActConns = 209
Returning 4 Connections
Allocating 3 Connections
cp.curActConns = 208
Returning 4 Connections
Allocating 0 Connections
cp.curActConns = 204
Returning 0 Connections
Allocating 0 Connections
cp.curActConns = 204
Returning 2 Connections
Allocating 3 Connections
cp.curActConns = 205
Returning 3 Connections
Allocating 2 Connections
cp.curActConns = 204
Returning 1 Connections
Allocating 0 Connections
cp.curActConns = 203
Returning 1 Connections
Allocating 4 Connections
cp.curActConns = 206
Returning 2 Connections
Allocating 1 Connections
cp.curActConns = 205
Returning 1 Connections
Allocating 3 Connections
cp.curActConns = 207
Returning 0 Connections
Allocating 1 Connections
cp.curActConns = 207
Returning 2 Connections
cp.curActConns = 206
Allocating 3 Connections
Returning 3 Connections
cp.curActConns = 206
Allocating 4 Connections
Returning 1 Connections
cp.curActConns = 209
Returning 2 Connections
Allocating 1 Connections
cp.curActConns = 208
Returning 3 Connections
Allocating 2 Connections
cp.curActConns = 207
Returning 3 Connections
Allocating 4 Connections
cp.curActConns = 208
Returning 1 Connections
Allocating 2 Connections
cp.curActConns = 209
Returning 2 Connections
Allocating 3 Connections
cp.curActConns = 210
Returning 1 Connections
Allocating 1 Connections
cp.curActConns = 210
Returning 3 Connections
Allocating 0 Connections
cp.curActConns = 207
Returning 1 Connections
Allocating 2 Connections
cp.curActConns = 208
Returning 1 Connections
Allocating 2 Connections
cp.curActConns = 209
Returning 4 Connections
Allocating 3 Connections
cp.curActConns = 208
Returning 0 Connections
Allocating 3 Connections
cp.curActConns = 211
Returning 2 Connections
Allocating 3 Connections
cp.curActConns = 212
Returning 3 Connections
Allocating 2 Connections
cp.curActConns = 211
Returning 0 Connections
Allocating 0 Connections
cp.curActConns = 211
Returning 4 Connections
Allocating 3 Connections
cp.curActConns = 210
Returning 4 Connections
Allocating 4 Connections
cp.curActConns = 210
Returning 2 Connections
Allocating 3 Connections
cp.curActConns = 211
Returning 4 Connections
Allocating 0 Connections
cp.curActConns = 207
Returning 0 Connections
Allocating 2 Connections
cp.curActConns = 209
Returning 2 Connections
Allocating 4 Connections
cp.curActConns = 211
Returning 2 Connections
Allocating 2 Connections
cp.curActConns = 211
Returning 4 Connections
Allocating 1 Connections
cp.curActConns = 208
Returning 1 Connections
Allocating 2 Connections
cp.curActConns = 209
Returning 1 Connections
Allocating 4 Connections
cp.curActConns = 212
Retuning from startDeallocatorRoutine
Retuning from startAllocatorRoutine
2023-09-01T13:16:05.400+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] ... stopped
2023-09-01T13:16:06.373+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] Stopping releaseConnsRoutine
--- PASS: TestSustainedHighConns (59.07s)
=== RUN   TestLowWM
2023-09-01T13:16:09.401+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] started poolsize 20 overflow 5 low WM 10 relConn batch size 2 ...
2023-09-01T13:17:09.421+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] active conns 0, free conns 10
2023-09-01T13:18:09.440+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] active conns 0, free conns 10
2023-09-01T13:18:14.905+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] ... stopped
2023-09-01T13:18:15.441+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] Stopping releaseConnsRoutine
--- PASS: TestLowWM (129.51s)
=== RUN   TestTotalConns
2023-09-01T13:18:18.907+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] started poolsize 120 overflow 5 low WM 10 relConn batch size 10 ...
2023-09-01T13:18:33.093+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] ... stopped
2023-09-01T13:18:33.914+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] Stopping releaseConnsRoutine
--- PASS: TestTotalConns (18.19s)
=== RUN   TestUpdateTickRate
2023-09-01T13:18:37.094+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] started poolsize 40 overflow 5 low WM 2 relConn batch size 2 ...
2023-09-01T13:18:57.932+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] ... stopped
2023-09-01T13:18:58.107+05:30 [Info] [Queryport-connpool:127.0.0.1:15151] Stopping releaseConnsRoutine
--- PASS: TestUpdateTickRate (24.84s)
PASS
ok  	github.com/couchbase/indexing/secondary/queryport/client	322.738s
Starting server: attempt 1

Functional tests

2023/09/01 13:21:31 In TestMain()
2023/09/01 13:21:31 otp node fetch error: json: cannot unmarshal string into Go value of type couchbase.Pool
2023/09/01 13:21:31 Initialising services with role: kv,n1ql on node: 127.0.0.1:9000
2023/09/01 13:21:32 Initialising web UI on node: 127.0.0.1:9000
2023/09/01 13:21:32 InitWebCreds, response is: {"newBaseUri":"http://127.0.0.1:9000/"}
2023/09/01 13:21:33 Setting data quota of 1500M and Index quota of 1500M
2023/09/01 13:21:34 Adding node: https://127.0.0.1:19001 with role: kv,index to the cluster
2023/09/01 13:21:50 AddNode: Successfully added node: 127.0.0.1:9001 (role kv,index), response: {"otpNode":"n_1@127.0.0.1"}
2023/09/01 13:21:55 Rebalance progress: 0
2023/09/01 13:22:00 Rebalance progress: 0
2023/09/01 13:22:05 Rebalance progress: 75
2023/09/01 13:22:10 Rebalance progress: 100
2023/09/01 13:22:15 Created bucket default, responseBody: 
2023/09/01 13:22:20 Cluster status: map[127.0.0.1:9001:[index kv] 172.31.5.112:9000:[kv n1ql]]
2023/09/01 13:22:20 Successfully initialised cluster
2023/09/01 13:22:20 Cluster status: map[127.0.0.1:9001:[index kv] 172.31.5.112:9000:[kv n1ql]]
2023/09/01 13:22:21 Changing config key queryport.client.settings.backfillLimit to value 0
2023/09/01 13:22:21 Changing config key queryport.client.log_level to value Warn
2023/09/01 13:22:21 Changing config key indexer.api.enableTestServer to value true
2023/09/01 13:22:21 Changing config key indexer.settings.persisted_snapshot_init_build.moi.interval to value 60000
2023/09/01 13:22:21 Changing config key indexer.settings.persisted_snapshot.moi.interval to value 60000
2023/09/01 13:22:21 Changing config key indexer.settings.log_level to value info
2023/09/01 13:22:21 Changing config key indexer.settings.storage_mode.disable_upgrade to value true
2023/09/01 13:22:21 Using plasma for creating indexes
2023/09/01 13:22:21 Changing config key indexer.settings.storage_mode to value plasma
2023/09/01 13:22:26 Data file exists. Skipping download
2023/09/01 13:22:26 Data file exists. Skipping download
2023/09/01 13:22:28 In DropAllSecondaryIndexes()
2023/09/01 13:22:28 Emptying the default bucket
2023/09/01 13:22:31 Flush Enabled on bucket default, responseBody: 
2023/09/01 13:23:09 Flushed the bucket default, Response body: 
2023/09/01 13:23:09 Create Index On the empty default Bucket()
2023/09/01 13:23:12 Created the secondary index index_eyeColor. Waiting for it become active
2023/09/01 13:23:12 Index is 776809365934153597 now active
2023/09/01 13:23:12 Populating the default bucket
=== RUN   TestScanAfterBucketPopulate
2023/09/01 13:23:22 In TestScanAfterBucketPopulate()
2023/09/01 13:23:22 Create an index on empty bucket, populate the bucket and Run a scan on the index
2023/09/01 13:23:22 Using n1ql client
2023-09-01T13:23:22.310+05:30 [Info] creating GsiClient for 127.0.0.1:9000
2023/09/01 13:23:22 Expected and Actual scan responses are the same
--- PASS: TestScanAfterBucketPopulate (0.10s)
=== RUN   TestRestartNilSnapshot
2023/09/01 13:23:22 In TestRestartNilSnapshot()
2023/09/01 13:23:26 Created the secondary index idx_age. Waiting for it become active
2023/09/01 13:23:26 Index is 14608988524843538081 now active
2023/09/01 13:23:26 Restarting indexer process ...
2023/09/01 13:23:27 []
2023-09-01T13:23:27.045+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2023-09-01T13:23:27.047+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2023/09/01 13:26:47 Using n1ql client
2023-09-01T13:26:47.019+05:30 [Error] transport error between 127.0.0.1:52612->127.0.0.1:9107: write tcp 127.0.0.1:52612->127.0.0.1:9107: write: broken pipe
2023-09-01T13:26:47.019+05:30 [Error] [GsiScanClient:"127.0.0.1:9107"] 8224434308349642664 request transport failed `write tcp 127.0.0.1:52612->127.0.0.1:9107: write: broken pipe`
2023-09-01T13:26:47.019+05:30 [Error] metadataClient:PickRandom: Fail to find indexer for all index partitions. Num partition 1.  Partition with instances 0 
2023-09-01T13:26:47.019+05:30 [Error] metadataClient:PickRandom: Replicas - [3214854434812111479], PrunedReplica - map[], FilteredReplica map[]
2023/09/01 13:26:47 Expected and Actual scan responses are the same
--- PASS: TestRestartNilSnapshot (204.67s)
=== RUN   TestThreeIndexCreates
2023/09/01 13:26:47 In TestThreeIndexCreates()
2023/09/01 13:26:51 Created the secondary index index_balance. Waiting for it become active
2023/09/01 13:26:51 Index is 8365539821886686861 now active
2023/09/01 13:26:51 Create docs mutations
2023/09/01 13:26:51 Using n1ql client
2023/09/01 13:26:51 Expected and Actual scan responses are the same
2023/09/01 13:26:58 Created the secondary index index_email. Waiting for it become active
2023/09/01 13:26:58 Index is 17457273494593145574 now active
2023/09/01 13:26:58 Create docs mutations
2023/09/01 13:26:58 Using n1ql client
2023/09/01 13:26:58 Expected and Actual scan responses are the same
2023/09/01 13:27:05 Created the secondary index index_pin. Waiting for it become active
2023/09/01 13:27:05 Index is 9658555280392182704 now active
2023/09/01 13:27:05 Delete docs mutations
2023/09/01 13:27:05 Using n1ql client
2023/09/01 13:27:05 Expected and Actual scan responses are the same
--- PASS: TestThreeIndexCreates (18.33s)
=== RUN   TestMultipleIndexCreatesDropsWithMutations
2023/09/01 13:27:05 In TestThreeIndexCreates()
2023/09/01 13:27:11 Created the secondary index index_state. Waiting for it become active
2023/09/01 13:27:11 Index is 4075537698929874469 now active
2023/09/01 13:27:11 Create docs mutations
2023/09/01 13:27:12 Using n1ql client
2023/09/01 13:27:12 Expected and Actual scan responses are the same
2023/09/01 13:27:18 Created the secondary index index_registered. Waiting for it become active
2023/09/01 13:27:18 Index is 510242736390324653 now active
2023/09/01 13:27:18 Create docs mutations
2023/09/01 13:27:18 Using n1ql client
2023/09/01 13:27:18 Expected and Actual scan responses are the same
2023/09/01 13:27:25 Created the secondary index index_gender. Waiting for it become active
2023/09/01 13:27:25 Index is 602687864570220189 now active
2023/09/01 13:27:25 Create docs mutations
2023/09/01 13:27:25 Using n1ql client
2023/09/01 13:27:25 Expected and Actual scan responses are the same
2023/09/01 13:27:25 Dropping the secondary index index_registered
2023/09/01 13:27:25 Index dropped
2023/09/01 13:27:25 Create docs mutations
2023/09/01 13:27:25 Delete docs mutations
2023/09/01 13:27:26 Using n1ql client
2023/09/01 13:27:26 Expected and Actual scan responses are the same
2023/09/01 13:27:31 Created the secondary index index_longitude. Waiting for it become active
2023/09/01 13:27:31 Index is 9679731895894679452 now active
2023/09/01 13:27:31 Create docs mutations
2023/09/01 13:27:31 Using n1ql client
2023/09/01 13:27:32 Expected and Actual scan responses are the same
--- PASS: TestMultipleIndexCreatesDropsWithMutations (26.62s)
=== RUN   TestCreateDropScan
2023/09/01 13:27:32 In TestCreateDropScan()
2023/09/01 13:27:38 Created the secondary index index_cd. Waiting for it become active
2023/09/01 13:27:38 Index is 13939787193944501463 now active
2023/09/01 13:27:38 Using n1ql client
2023/09/01 13:27:38 Expected and Actual scan responses are the same
2023/09/01 13:27:38 Dropping the secondary index index_cd
2023/09/01 13:27:38 Index dropped
2023/09/01 13:27:38 Using n1ql client
2023/09/01 13:27:38 Scan failed as expected with error: Index Not Found - cause: GSI index index_cd not found.
--- PASS: TestCreateDropScan (6.74s)
=== RUN   TestCreateDropCreate
2023/09/01 13:27:38 In TestCreateDropCreate()
2023/09/01 13:27:45 Created the secondary index index_cdc. Waiting for it become active
2023/09/01 13:27:45 Index is 5824552567752797176 now active
2023/09/01 13:27:45 Using n1ql client
2023/09/01 13:27:45 Expected and Actual scan responses are the same
2023/09/01 13:27:45 Dropping the secondary index index_cdc
2023/09/01 13:27:45 Index dropped
2023/09/01 13:27:45 Using n1ql client
2023/09/01 13:27:45 Scan 2 failed as expected with error: Index Not Found - cause: GSI index index_cdc not found.
2023/09/01 13:27:51 Created the secondary index index_cdc. Waiting for it become active
2023/09/01 13:27:51 Index is 7791829915709905297 now active
2023/09/01 13:27:51 Using n1ql client
2023/09/01 13:27:51 Expected and Actual scan responses are the same
2023/09/01 13:27:51 (Inclusion 1) Lengths of expected and actual scan results are 5042 and 5042. Num of docs in bucket = 10402
2023/09/01 13:27:51 Using n1ql client
2023/09/01 13:27:51 Expected and Actual scan responses are the same
2023/09/01 13:27:51 (Inclusion 3) Lengths of expected and actual scan results are 5042 and 5042. Num of docs in bucket = 10402
--- PASS: TestCreateDropCreate (13.13s)
=== RUN   TestCreate2Drop1Scan2
2023/09/01 13:27:51 In TestCreate2Drop1Scan2()
2023/09/01 13:27:58 Created the secondary index index_i1. Waiting for it become active
2023/09/01 13:27:58 Index is 15995283817920644825 now active
2023/09/01 13:28:04 Created the secondary index index_i2. Waiting for it become active
2023/09/01 13:28:04 Index is 14791940299285780894 now active
2023/09/01 13:28:04 Using n1ql client
2023/09/01 13:28:04 Expected and Actual scan responses are the same
2023/09/01 13:28:04 Using n1ql client
2023/09/01 13:28:04 Expected and Actual scan responses are the same
2023/09/01 13:28:04 Dropping the secondary index index_i1
2023/09/01 13:28:05 Index dropped
2023/09/01 13:28:05 Using n1ql client
2023/09/01 13:28:05 Expected and Actual scan responses are the same
--- PASS: TestCreate2Drop1Scan2 (13.26s)
=== RUN   TestIndexNameCaseSensitivity
2023/09/01 13:28:05 In TestIndexNameCaseSensitivity()
2023/09/01 13:28:11 Created the secondary index index_age. Waiting for it become active
2023/09/01 13:28:11 Index is 18025153882317709445 now active
2023/09/01 13:28:11 Using n1ql client
2023/09/01 13:28:11 Expected and Actual scan responses are the same
2023/09/01 13:28:11 Using n1ql client
2023/09/01 13:28:11 Scan failed as expected with error: Index Not Found - cause: GSI index index_Age not found.
--- PASS: TestIndexNameCaseSensitivity (6.38s)
=== RUN   TestCreateDuplicateIndex
2023/09/01 13:28:11 In TestCreateDuplicateIndex()
2023/09/01 13:28:17 Created the secondary index index_di1. Waiting for it become active
2023/09/01 13:28:17 Index is 12806325552594474781 now active
2023/09/01 13:28:17 Index found:  index_di1
2023/09/01 13:28:17 Create failed as expected with error: Index index_di1 already exists.
--- PASS: TestCreateDuplicateIndex (6.37s)
=== RUN   TestDropNonExistingIndex
2023/09/01 13:28:17 In TestDropNonExistingIndex()
2023/09/01 13:28:17 Dropping the secondary index 123456
2023/09/01 13:28:17 Index drop failed as expected with error: Index does not exist.
--- PASS: TestDropNonExistingIndex (0.07s)
=== RUN   TestCreateIndexNonExistentBucket
2023/09/01 13:28:17 In TestCreateIndexNonExistentBucket()
2023-09-01T13:28:18.463+05:30 [Error] Encountered error during create index.  Error: [Bucket Not Found] Bucket BlahBucket does not exist or temporarily unavailable for creating new index. Please retry the operation at a later time.
2023-09-01T13:28:29.466+05:30 [Error] Fail to create index: [Bucket Not Found] Bucket BlahBucket does not exist or temporarily unavailable for creating new index. Please retry the operation at a later time.
2023/09/01 13:28:29 Index create failed as expected with error: [Bucket Not Found] Bucket BlahBucket does not exist or temporarily unavailable for creating new index. Please retry the operation at a later time.
--- PASS: TestCreateIndexNonExistentBucket (11.51s)
=== RUN   TestScanWithNoTimeout
2023/09/01 13:28:29 Create an index on empty bucket, populate the bucket and Run a scan on the index
2023/09/01 13:28:29 Changing config key indexer.settings.scan_timeout to value 0
2023/09/01 13:28:29 Using n1ql client
2023/09/01 13:28:29 Expected and Actual scan responses are the same
--- PASS: TestScanWithNoTimeout (0.41s)
=== RUN   TestIndexingOnBinaryBucketMeta
2023/09/01 13:28:29 In TestIndexingOnBinaryBucketMeta()
2023/09/01 13:28:29 	 1. Populate a bucekt with binary docs and create indexs on the `id`, `cas` and `expiration` fields of Metadata
2023/09/01 13:28:29 	 2. Validate the test by comparing the items_count of indexes and the number of docs in the bucket for each of the fields
2023/09/01 13:28:32 Modified parameters of bucket default, responseBody: 
2023/09/01 13:28:33 Created bucket binaryBucket, responseBody: 
2023/09/01 13:28:51 Created the secondary index index_binary_meta_id. Waiting for it become active
2023/09/01 13:28:51 Index is 4436727442065270875 now active
2023/09/01 13:28:56 items_count stat is 10 for index index_binary_meta_id
2023/09/01 13:28:56 Dropping the secondary index index_binary_meta_id
2023/09/01 13:28:56 Index dropped
2023/09/01 13:28:59 Created the secondary index index_binary_meta_cas. Waiting for it become active
2023/09/01 13:28:59 Index is 18425139961255462964 now active
2023/09/01 13:29:04 items_count stat is 10 for index index_binary_meta_cas
2023/09/01 13:29:04 Dropping the secondary index index_binary_meta_cas
2023/09/01 13:29:04 Index dropped
2023/09/01 13:29:07 Created the secondary index index_binary_meta_expiration. Waiting for it become active
2023/09/01 13:29:07 Index is 10505391068509731175 now active
2023/09/01 13:29:12 items_count stat is 10 for index index_binary_meta_expiration
2023/09/01 13:29:12 Dropping the secondary index index_binary_meta_expiration
2023/09/01 13:29:12 Index dropped
2023/09/01 13:29:14 Deleted bucket binaryBucket, responseBody: 
2023/09/01 13:29:17 Modified parameters of bucket default, responseBody: 
--- PASS: TestIndexingOnBinaryBucketMeta (63.10s)
=== RUN   TestRetainDeleteXATTRBinaryDocs
2023/09/01 13:29:32 In TestRetainDeleteXATTRBinaryDocs()
2023/09/01 13:29:32 	 1. Populate a bucket with binary docs having system XATTRS
2023/09/01 13:29:32 	 2. Create index on the system XATTRS with "retain_deleted_xattr" attribute set to true
2023/09/01 13:29:32 	 3. Delete the documents in the bucket
2023/09/01 13:29:32 	 4. Query for the meta() information in the source bucket. The total number of results should be equivalent to the number of documents in the bucket before deletion of documents
2023/09/01 13:29:36 Modified parameters of bucket default, responseBody: 
2023/09/01 13:29:36 Created bucket binaryBucket, responseBody: 
2023/09/01 13:29:54 Created the secondary index index_system_xattr. Waiting for it become active
2023/09/01 13:29:54 Index is 1963403771263103352 now active
2023/09/01 13:29:59 Deleted all the documents in the bucket: binaryBucket successfully
2023/09/01 13:30:02 Deleted bucket binaryBucket, responseBody: 
2023/09/01 13:30:05 Modified parameters of bucket default, responseBody: 
--- PASS: TestRetainDeleteXATTRBinaryDocs (47.56s)
=== RUN   TestIndexingOnXATTRs
2023/09/01 13:30:20 In TestIndexingOnXATTRs()
2023/09/01 13:30:23 Modified parameters of bucket default, responseBody: 
2023/09/01 13:30:23 Created bucket bucket_xattrs, responseBody: 
2023/09/01 13:30:43 Created the secondary index index_sync_rev. Waiting for it become active
2023/09/01 13:30:43 Index is 18089343515255180857 now active
2023/09/01 13:30:50 Created the secondary index index_sync_channels. Waiting for it become active
2023/09/01 13:30:50 Index is 9392557714430940836 now active
2023/09/01 13:30:50 Creating index index_sync_sequence. Error while parsing the expression (meta().xattrs._sync.sequence) : syntax error - line 1, column 21, near 'meta().xattrs._sync.', at: sequence (reserved word)
    common_test.go:173: Error creatingindex_sync_sequence on xattr docs: syntax error - line 1, column 21, near 'meta().xattrs._sync.', at: sequence (reserved word)
--- FAIL: TestIndexingOnXATTRs (29.58s)
=== RUN   TestSimpleIndex_FloatDataType
2023/09/01 13:30:50 In TestSimpleIndex_FloatDataType()
2023/09/01 13:30:50 Index found:  index_age
2023/09/01 13:30:50 Using n1ql client
2023/09/01 13:30:50 Expected and Actual scan responses are the same
--- PASS: TestSimpleIndex_FloatDataType (0.08s)
=== RUN   TestSimpleIndex_StringDataType
2023/09/01 13:30:50 In TestSimpleIndex_StringDataType()
2023/09/01 13:30:56 Created the secondary index index_company. Waiting for it become active
2023/09/01 13:30:56 Index is 9957814846660671185 now active
2023/09/01 13:30:56 Using n1ql client
2023/09/01 13:30:56 Expected and Actual scan responses are the same
2023/09/01 13:30:56 Using n1ql client
2023/09/01 13:30:57 Expected and Actual scan responses are the same
--- PASS: TestSimpleIndex_StringDataType (6.87s)
=== RUN   TestSimpleIndex_FieldValueCaseSensitivity
2023/09/01 13:30:57 In TestSimpleIndex_StringCaseSensitivity()
2023/09/01 13:30:57 Index found:  index_company
2023/09/01 13:30:57 Using n1ql client
2023/09/01 13:30:57 Expected and Actual scan responses are the same
2023/09/01 13:30:57 Using n1ql client
2023/09/01 13:30:57 Expected and Actual scan responses are the same
--- PASS: TestSimpleIndex_FieldValueCaseSensitivity (0.07s)
=== RUN   TestSimpleIndex_BoolDataType
2023/09/01 13:30:57 In TestSimpleIndex_BoolDataType()
2023/09/01 13:31:03 Created the secondary index index_isActive. Waiting for it become active
2023/09/01 13:31:03 Index is 11303188647282878847 now active
2023/09/01 13:31:03 Using n1ql client
2023/09/01 13:31:03 Expected and Actual scan responses are the same
--- PASS: TestSimpleIndex_BoolDataType (6.87s)
=== RUN   TestBasicLookup
2023/09/01 13:31:03 In TestBasicLookup()
2023/09/01 13:31:03 Index found:  index_company
2023/09/01 13:31:04 Using n1ql client
2023/09/01 13:31:04 Expected and Actual scan responses are the same
--- PASS: TestBasicLookup (0.02s)
=== RUN   TestIndexOnNonExistentField
2023/09/01 13:31:04 In TestIndexOnNonExistentField()
2023/09/01 13:31:10 Created the secondary index index_height. Waiting for it become active
2023/09/01 13:31:10 Index is 18417139391721591776 now active
2023/09/01 13:31:10 Using n1ql client
2023/09/01 13:31:10 Expected and Actual scan responses are the same
--- PASS: TestIndexOnNonExistentField (6.73s)
=== RUN   TestIndexPartiallyMissingField
2023/09/01 13:31:10 In TestIndexPartiallyMissingField()
2023/09/01 13:31:17 Created the secondary index index_nationality. Waiting for it become active
2023/09/01 13:31:17 Index is 8769571713881325382 now active
2023/09/01 13:31:17 Using n1ql client
2023/09/01 13:31:17 Expected and Actual scan responses are the same
--- PASS: TestIndexPartiallyMissingField (6.69s)
=== RUN   TestScanNonMatchingDatatype
2023/09/01 13:31:17 In TestScanNonMatchingDatatype()
2023/09/01 13:31:17 Index found:  index_age
2023/09/01 13:31:17 Using n1ql client
2023/09/01 13:31:17 Expected and Actual scan responses are the same
--- PASS: TestScanNonMatchingDatatype (0.01s)
=== RUN   TestInclusionNeither
2023/09/01 13:31:17 In TestInclusionNeither()
2023/09/01 13:31:17 Index found:  index_age
2023/09/01 13:31:17 Using n1ql client
2023/09/01 13:31:17 Expected and Actual scan responses are the same
--- PASS: TestInclusionNeither (0.05s)
=== RUN   TestInclusionLow
2023/09/01 13:31:17 In TestInclusionLow()
2023/09/01 13:31:17 Index found:  index_age
2023/09/01 13:31:17 Using n1ql client
2023/09/01 13:31:17 Expected and Actual scan responses are the same
--- PASS: TestInclusionLow (0.04s)
=== RUN   TestInclusionHigh
2023/09/01 13:31:17 In TestInclusionHigh()
2023/09/01 13:31:17 Index found:  index_age
2023/09/01 13:31:17 Using n1ql client
2023/09/01 13:31:17 Expected and Actual scan responses are the same
--- PASS: TestInclusionHigh (0.03s)
=== RUN   TestInclusionBoth
2023/09/01 13:31:17 In TestInclusionBoth()
2023/09/01 13:31:17 Index found:  index_age
2023/09/01 13:31:17 Using n1ql client
2023/09/01 13:31:17 Expected and Actual scan responses are the same
--- PASS: TestInclusionBoth (0.03s)
=== RUN   TestNestedIndex_String
2023/09/01 13:31:17 In TestNestedIndex_String()
2023/09/01 13:31:24 Created the secondary index index_streetname. Waiting for it become active
2023/09/01 13:31:24 Index is 2255246268639412273 now active
2023/09/01 13:31:24 Using n1ql client
2023/09/01 13:31:24 Expected and Actual scan responses are the same
--- PASS: TestNestedIndex_String (6.59s)
=== RUN   TestNestedIndex_Float
2023/09/01 13:31:24 In TestNestedIndex_Float()
2023/09/01 13:31:30 Created the secondary index index_floor. Waiting for it become active
2023/09/01 13:31:30 Index is 7467198430100054424 now active
2023/09/01 13:31:30 Using n1ql client
2023/09/01 13:31:30 Expected and Actual scan responses are the same
--- PASS: TestNestedIndex_Float (6.75s)
=== RUN   TestNestedIndex_Bool
2023/09/01 13:31:30 In TestNestedIndex_Bool()
2023/09/01 13:31:37 Created the secondary index index_isresidential. Waiting for it become active
2023/09/01 13:31:37 Index is 1277945552392373442 now active
2023/09/01 13:31:37 Using n1ql client
2023/09/01 13:31:37 Expected and Actual scan responses are the same
--- PASS: TestNestedIndex_Bool (6.49s)
=== RUN   TestLookupJsonObject
2023/09/01 13:31:37 In TestLookupJsonObject()
2023/09/01 13:31:44 Created the secondary index index_streetaddress. Waiting for it become active
2023/09/01 13:31:44 Index is 10295813826686457183 now active
2023/09/01 13:31:44 Using n1ql client
2023/09/01 13:31:44 Count of docScanResults is 1
2023/09/01 13:31:44 Key: User3bf51f08-0bac-4c03-bcec-5c255cbdde2c  Value: [map[buildingname:Sterling Heights doornumber:12B floor:5 streetname:Hill Street]]
2023/09/01 13:31:44 Count of scanResults is 1
2023/09/01 13:31:44 Key: string User3bf51f08-0bac-4c03-bcec-5c255cbdde2c  Value: value.Values [{"buildingname":"Sterling Heights","doornumber":"12B","floor":5,"streetname":"Hill Street"}] false
2023/09/01 13:31:44 Expected and Actual scan responses are the same
--- PASS: TestLookupJsonObject (6.77s)
=== RUN   TestLookupObjDifferentOrdering
2023/09/01 13:31:44 In TestLookupObjDifferentOrdering()
2023/09/01 13:31:44 Index found:  index_streetaddress
2023/09/01 13:31:44 Using n1ql client
2023/09/01 13:31:44 Count of docScanResults is 1
2023/09/01 13:31:44 Key: User3bf51f08-0bac-4c03-bcec-5c255cbdde2c  Value: [map[buildingname:Sterling Heights doornumber:12B floor:5 streetname:Hill Street]]
2023/09/01 13:31:44 Count of scanResults is 1
2023/09/01 13:31:44 Key: string User3bf51f08-0bac-4c03-bcec-5c255cbdde2c  Value: value.Values [{"buildingname":"Sterling Heights","doornumber":"12B","floor":5,"streetname":"Hill Street"}] false
2023/09/01 13:31:44 Expected and Actual scan responses are the same
--- PASS: TestLookupObjDifferentOrdering (0.01s)
=== RUN   TestRangeJsonObject
2023/09/01 13:31:44 In TestRangeJsonObject()
2023/09/01 13:31:44 Index found:  index_streetaddress
2023/09/01 13:31:44 Using n1ql client
2023/09/01 13:31:44 Count of scanResults is 2
2023/09/01 13:31:44 Key: string Userbb48952f-f8d1-4e04-a0e1-96b9019706fb  Value: value.Values [{"buildingname":"Rosewood Gardens","doornumber":"514","floor":2,"streetname":"Karweg Place"}] false
2023/09/01 13:31:44 Key: string User3bf51f08-0bac-4c03-bcec-5c255cbdde2c  Value: value.Values [{"buildingname":"Sterling Heights","doornumber":"12B","floor":5,"streetname":"Hill Street"}] false
2023/09/01 13:31:44 Count of docScanResults is 2
2023/09/01 13:31:44 Key: User3bf51f08-0bac-4c03-bcec-5c255cbdde2c  Value: [map[buildingname:Sterling Heights doornumber:12B floor:5 streetname:Hill Street]]
2023/09/01 13:31:44 Key: Userbb48952f-f8d1-4e04-a0e1-96b9019706fb  Value: [map[buildingname:Rosewood Gardens doornumber:514 floor:2 streetname:Karweg Place]]
2023/09/01 13:31:44 Expected and Actual scan responses are the same
--- PASS: TestRangeJsonObject (0.00s)
=== RUN   TestLookupFloatDiffForms
2023/09/01 13:31:44 In TestLookupFloatDiffForms()
2023/09/01 13:31:50 Created the secondary index index_latitude. Waiting for it become active
2023/09/01 13:31:50 Index is 7729150400900276302 now active
2023/09/01 13:31:50 Scan 1
2023/09/01 13:31:51 Using n1ql client
2023/09/01 13:31:51 Expected and Actual scan responses are the same
2023/09/01 13:31:51 Scan 2
2023/09/01 13:31:51 Using n1ql client
2023/09/01 13:31:51 Expected and Actual scan responses are the same
2023/09/01 13:31:51 Scan 3
2023/09/01 13:31:51 Using n1ql client
2023/09/01 13:31:51 Expected and Actual scan responses are the same
2023/09/01 13:31:51 Scan 4
2023/09/01 13:31:51 Using n1ql client
2023/09/01 13:31:51 Expected and Actual scan responses are the same
2023/09/01 13:31:51 Scan 5
2023/09/01 13:31:51 Using n1ql client
2023/09/01 13:31:51 Expected and Actual scan responses are the same
2023/09/01 13:31:51 Scan 6
2023/09/01 13:31:51 Using n1ql client
2023/09/01 13:31:51 Expected and Actual scan responses are the same
--- PASS: TestLookupFloatDiffForms (6.92s)
=== RUN   TestRangeFloatInclVariations
2023/09/01 13:31:51 In TestRangeFloatInclVariations()
2023/09/01 13:31:51 Index found:  index_latitude
2023/09/01 13:31:51 Scan 1
2023/09/01 13:31:51 Using n1ql client
2023/09/01 13:31:51 Expected and Actual scan responses are the same
2023/09/01 13:31:51 Scan 2
2023/09/01 13:31:51 Using n1ql client
2023/09/01 13:31:51 Expected and Actual scan responses are the same
2023/09/01 13:31:51 Scan 3
2023/09/01 13:31:51 Using n1ql client
2023/09/01 13:31:51 Expected and Actual scan responses are the same
2023/09/01 13:31:51 Scan 4
2023/09/01 13:31:51 Using n1ql client
2023/09/01 13:31:51 Expected and Actual scan responses are the same
2023/09/01 13:31:51 Scan 5
2023/09/01 13:31:51 Using n1ql client
2023/09/01 13:31:51 Expected and Actual scan responses are the same
2023/09/01 13:31:51 Scan 6
2023/09/01 13:31:51 Using n1ql client
2023/09/01 13:31:51 Expected and Actual scan responses are the same
--- PASS: TestRangeFloatInclVariations (0.03s)
=== RUN   TestScanAll
2023/09/01 13:31:51 In TestScanAll()
2023/09/01 13:31:57 Created the secondary index index_name. Waiting for it become active
2023/09/01 13:31:57 Index is 5713130578488721421 now active
2023/09/01 13:31:57 Length of docScanResults = 10502
2023/09/01 13:31:57 Using n1ql client
2023/09/01 13:31:57 Length of scanResults = 10502
2023/09/01 13:31:58 Expected and Actual scan responses are the same
--- PASS: TestScanAll (6.86s)
=== RUN   TestScanAllNestedField
2023/09/01 13:31:58 In TestScanAllNestedField()
2023/09/01 13:31:58 Index found:  index_streetname
2023/09/01 13:31:58 Length of docScanResults = 2
2023/09/01 13:31:58 Using n1ql client
2023/09/01 13:31:58 Length of scanResults = 2
2023/09/01 13:31:58 Expected and Actual scan responses are the same
--- PASS: TestScanAllNestedField (0.01s)
=== RUN   TestBasicPrimaryIndex
2023/09/01 13:31:58 In TestBasicPrimaryIndex()
2023/09/01 13:32:04 Created the secondary index index_p1. Waiting for it become active
2023/09/01 13:32:04 Index is 13430922790460519611 now active
2023-09-01T13:32:04.349+05:30 [Error] transport error between 127.0.0.1:52276->127.0.0.1:9107: write tcp 127.0.0.1:52276->127.0.0.1:9107: write: broken pipe
2023-09-01T13:32:04.349+05:30 [Error] [GsiScanClient:"127.0.0.1:9107"]  request transport failed `write tcp 127.0.0.1:52276->127.0.0.1:9107: write: broken pipe`
2023-09-01T13:32:04.349+05:30 [Error] metadataClient:PickRandom: Fail to find indexer for all index partitions. Num partition 1.  Partition with instances 0 
2023-09-01T13:32:04.349+05:30 [Error] metadataClient:PickRandom: Replicas - [6552902241767412067], PrunedReplica - map[], FilteredReplica map[]
2023/09/01 13:32:04 Expected and Actual scan responses are the same
2023/09/01 13:32:04 CountRange() expected and actual is:  1882 and 1882
2023/09/01 13:32:04 lookupkey for CountLookup() = Userfc11554c-b4cc-4c7a-a031-79572799a856
2023/09/01 13:32:04 CountLookup() = 1
--- PASS: TestBasicPrimaryIndex (6.44s)
=== RUN   TestBasicNullDataType
2023/09/01 13:32:04 In TestBasicNullDataType()
2023/09/01 13:32:04 Index found:  index_email
2023/09/01 13:32:04 Using n1ql client
2023/09/01 13:32:04 Expected and Actual scan responses are the same
--- PASS: TestBasicNullDataType (0.01s)
=== RUN   TestBasicArrayDataType_ScanAll
2023/09/01 13:32:04 In TestBasicArrayDataType_ScanAll()
2023/09/01 13:32:11 Created the secondary index index_tags. Waiting for it become active
2023/09/01 13:32:11 Index is 16692924923819297735 now active
2023/09/01 13:32:11 Using n1ql client
2023/09/01 13:32:11 Expected and Actual scan responses are the same
--- PASS: TestBasicArrayDataType_ScanAll (7.11s)
=== RUN   TestBasicArrayDataType_Lookup
2023/09/01 13:32:11 In TestBasicArrayDataType_Lookup()
2023/09/01 13:32:13 Index found:  index_tags
2023/09/01 13:32:13 Count of scanResults is 1
2023/09/01 13:32:13 Key: string Usere46cea01-38f6-4e7b-92e5-69d64668ae75  Value: value.Values [["reprehenderit","tempor","officia","exercitation","labore","sunt","tempor"]] false
--- PASS: TestBasicArrayDataType_Lookup (2.00s)
=== RUN   TestArrayDataType_LookupMissingArrayValue
2023/09/01 13:32:13 In TestArrayDataType_LookupMissingArrayValue()
2023/09/01 13:32:13 Index found:  index_tags
2023/09/01 13:32:13 Count of scanResults is 0
--- PASS: TestArrayDataType_LookupMissingArrayValue (0.00s)
=== RUN   TestArrayDataType_LookupWrongOrder
2023/09/01 13:32:13 In TestArrayDataType_LookupWrongOrder()
2023/09/01 13:32:13 Index found:  index_tags
2023/09/01 13:32:13 Count of scanResults is 0
--- PASS: TestArrayDataType_LookupWrongOrder (0.00s)
=== RUN   TestArrayDataType_LookupSubset
2023/09/01 13:32:13 In TestArrayDataType_LookupSubset()
2023/09/01 13:32:13 Index found:  index_tags
2023/09/01 13:32:13 Count of scanResults is 0
--- PASS: TestArrayDataType_LookupSubset (0.00s)
=== RUN   TestScanLimitParameter
2023/09/01 13:32:13 In TestScanLimitParameter()
2023/09/01 13:32:13 Index found:  index_age
2023/09/01 13:32:13 Using n1ql client
2023/09/01 13:32:13 Using n1ql client
--- PASS: TestScanLimitParameter (0.01s)
=== RUN   TestCountRange
2023/09/01 13:32:13 In TestRangeCount()
2023/09/01 13:32:13 Index found:  index_age
2023/09/01 13:32:13 Count of expected and actual Range is:  2381 and 2381
2023/09/01 13:32:13 Count of expected and actual Range is: 10002 and 10002
2023/09/01 13:32:13 Count of expected and actual Range are: 0 and 0
2023/09/01 13:32:13 Count of expected and actual Range are: 483 and 483
2023/09/01 13:32:13 Testing CountRange() for key <= val
2023/09/01 13:32:13 Count of expected and actual CountRange for key <= 30 are: 5229 and 5229
2023/09/01 13:32:13 Testing CountRange() for key >= val
2023/09/01 13:32:13 Count of expected and actual CountRange for key >= 25 are: 7684 and 7684
2023/09/01 13:32:13 Testing CountRange() for null < key <= val
2023/09/01 13:32:13 Count of expected and actual CountRange for key > null && key <= 30 are: 5229 and 5229
2023/09/01 13:32:13 Testing CountRange() for val <= key < null 
2023/09/01 13:32:13 Count of expected and actual CountRange for key >= 25 && key < null are: 0 and 0
2023/09/01 13:32:13 Count of expected and actual Range are: 0 and 0
--- PASS: TestCountRange (0.06s)
=== RUN   TestCountLookup
2023/09/01 13:32:13 In TestCountLookup()
2023/09/01 13:32:13 Index found:  index_age
2023/09/01 13:32:13 Count of expected and actual Range are: 502 and 502
2023/09/01 13:32:13 Count of expected and actual Range are: 0 and 0
--- PASS: TestCountLookup (0.01s)
=== RUN   TestRangeStatistics
2023/09/01 13:32:13 In TestRangeCount()
2023/09/01 13:32:13 Index found:  index_age
--- PASS: TestRangeStatistics (0.00s)
=== RUN   TestIndexCreateWithWhere
2023/09/01 13:32:13 In TestIndexCreateWithWhere()
2023/09/01 13:32:18 Created the secondary index index_ageabove30. Waiting for it become active
2023/09/01 13:32:18 Index is 15635420728825813075 now active
2023/09/01 13:32:18 Using n1ql client
2023/09/01 13:32:18 Expected and Actual scan responses are the same
2023/09/01 13:32:18 Lengths of expected and actual scanReuslts are:  4290 and 4290
2023/09/01 13:32:25 Created the secondary index index_ageteens. Waiting for it become active
2023/09/01 13:32:25 Index is 18152446424823749313 now active
2023/09/01 13:32:25 Using n1ql client
2023/09/01 13:32:26 Expected and Actual scan responses are the same
2023/09/01 13:32:26 Lengths of expected and actual scanReuslts are:  0 and 0
2023/09/01 13:32:31 Created the secondary index index_age35to45. Waiting for it become active
2023/09/01 13:32:31 Index is 3736982701688478538 now active
2023/09/01 13:32:31 Using n1ql client
2023/09/01 13:32:31 Expected and Actual scan responses are the same
2023/09/01 13:32:31 Lengths of expected and actual scanReuslts are:  2864 and 2864
--- PASS: TestIndexCreateWithWhere (18.19s)
=== RUN   TestDeferredIndexCreate
2023/09/01 13:32:31 In TestDeferredIndexCreate()
2023/09/01 13:32:32 Created the index index_deferred in deferred mode. Index state is INDEX_STATE_READY
2023/09/01 13:32:34 Build the deferred index index_deferred. Waiting for the index to become active
2023/09/01 13:32:34 Waiting for index 5023752718770665612 to go active ...
2023/09/01 13:32:35 Waiting for index 5023752718770665612 to go active ...
2023/09/01 13:32:36 Waiting for index 5023752718770665612 to go active ...
2023/09/01 13:32:37 Waiting for index 5023752718770665612 to go active ...
2023/09/01 13:32:38 Waiting for index 5023752718770665612 to go active ...
2023/09/01 13:32:39 Index is 5023752718770665612 now active
2023/09/01 13:32:39 Using n1ql client
2023/09/01 13:32:39 Expected and Actual scan responses are the same
--- PASS: TestDeferredIndexCreate (7.23s)
=== RUN   TestCompositeIndex_NumAndString
2023/09/01 13:32:39 In TestCompositeIndex()
2023/09/01 13:32:46 Created the secondary index index_composite1. Waiting for it become active
2023/09/01 13:32:46 Index is 16110673065399727593 now active
2023/09/01 13:32:46 Using n1ql client
2023/09/01 13:32:46 Using n1ql client
2023/09/01 13:32:46 Using n1ql client
2023/09/01 13:32:46 Expected and Actual scan responses are the same
--- PASS: TestCompositeIndex_NumAndString (7.08s)
=== RUN   TestCompositeIndex_TwoNumberFields
2023/09/01 13:32:46 In TestCompositeIndex()
2023/09/01 13:32:52 Created the secondary index index_composite2. Waiting for it become active
2023/09/01 13:32:52 Index is 13260896349836312911 now active
2023/09/01 13:32:52 Using n1ql client
--- PASS: TestCompositeIndex_TwoNumberFields (6.98s)
=== RUN   TestNumbers_Int64_Float64
2023/09/01 13:32:53 In TestNumbers_Int64_Float64()
2023/09/01 13:32:59 Created the secondary index idx_numbertest. Waiting for it become active
2023/09/01 13:32:59 Index is 8442687334618525478 now active
2023/09/01 13:32:59 
 ==== Int64 test #0
2023/09/01 13:32:59 Using n1ql client
2023/09/01 13:32:59 Expected and Actual scan responses are the same
2023/09/01 13:32:59 
 ==== Int64 test #1
2023/09/01 13:32:59 Using n1ql client
2023/09/01 13:32:59 Expected and Actual scan responses are the same
2023/09/01 13:32:59 
 ==== Int64 test #2
2023/09/01 13:32:59 Using n1ql client
2023/09/01 13:32:59 Expected and Actual scan responses are the same
2023/09/01 13:32:59 
 ==== Int64 test #3
2023/09/01 13:32:59 Using n1ql client
2023/09/01 13:32:59 Expected and Actual scan responses are the same
2023/09/01 13:32:59 
 ==== Int64 test #4
2023/09/01 13:33:00 Using n1ql client
2023/09/01 13:33:00 Expected and Actual scan responses are the same
2023/09/01 13:33:00 
 ==== Int64 test #5
2023/09/01 13:33:00 Using n1ql client
2023/09/01 13:33:00 Expected and Actual scan responses are the same
2023/09/01 13:33:00 
 ==== Int64 test #6
2023/09/01 13:33:00 Using n1ql client
2023/09/01 13:33:00 Expected and Actual scan responses are the same
2023/09/01 13:33:00 
 ==== Int64 test #7
2023/09/01 13:33:00 Using n1ql client
2023/09/01 13:33:00 Expected and Actual scan responses are the same
2023/09/01 13:33:00 
 ==== Int64 test #8
2023/09/01 13:33:00 Using n1ql client
2023/09/01 13:33:00 Expected and Actual scan responses are the same
2023/09/01 13:33:00 
 ==== Float64 test #0
2023/09/01 13:33:00 Using n1ql client
2023/09/01 13:33:00 Expected and Actual scan responses are the same
2023/09/01 13:33:00 
 ==== Float64 test #1
2023/09/01 13:33:00 Using n1ql client
2023/09/01 13:33:00 Expected and Actual scan responses are the same
2023/09/01 13:33:00 
 ==== Float64 test #2
2023/09/01 13:33:00 Using n1ql client
2023/09/01 13:33:00 Expected and Actual scan responses are the same
2023/09/01 13:33:00 
 ==== Float64 test #3
2023/09/01 13:33:00 Using n1ql client
2023/09/01 13:33:00 Expected and Actual scan responses are the same
--- PASS: TestNumbers_Int64_Float64 (7.29s)
=== RUN   TestRestartIndexer
2023/09/01 13:33:00 In TestRestartIndexer()
2023/09/01 13:33:00 []
2023-09-01T13:33:00.542+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2023-09-01T13:33:00.542+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly.
2023/09/01 13:33:20 Using n1ql client
2023-09-01T13:33:20.481+05:30 [Error] transport error between 127.0.0.1:53156->127.0.0.1:9107: write tcp 127.0.0.1:53156->127.0.0.1:9107: write: broken pipe
2023-09-01T13:33:20.481+05:30 [Error] [GsiScanClient:"127.0.0.1:9107"] 7725537561251029566 request transport failed `write tcp 127.0.0.1:53156->127.0.0.1:9107: write: broken pipe`
2023/09/01 13:33:20 Len of expected and actual scan results are :  10002 and 10002
2023/09/01 13:33:20 Expected and Actual scan responses are the same
--- PASS: TestRestartIndexer (20.09s)
=== RUN   TestCreateDocsMutation
2023/09/01 13:33:20 In TestCreateDocsMutation()
2023/09/01 13:33:20 Index found:  index_age
2023/09/01 13:33:20 Using n1ql client
2023/09/01 13:33:20 Len of expected and actual scan results are :  10002 and 10002
2023/09/01 13:33:20 Expected and Actual scan responses are the same
2023/09/01 13:33:20 Using n1ql client
2023/09/01 13:33:21 Index Scan after mutations took 253.231928ms
2023/09/01 13:33:21 Len of expected and actual scan results are :  10102 and 10102
2023/09/01 13:33:21 Expected and Actual scan responses are the same
--- PASS: TestCreateDocsMutation (0.59s)
=== RUN   TestRestartProjector
2023/09/01 13:33:21 In TestRestartProjector()
2023/09/01 13:33:21 []
2023/09/01 13:33:41 Using n1ql client
2023/09/01 13:33:41 Len of expected and actual scan results are :  10102 and 10102
2023/09/01 13:33:41 Expected and Actual scan responses are the same
--- PASS: TestRestartProjector (20.08s)
=== RUN   TestDeleteDocsMutation
2023/09/01 13:33:41 In TestDeleteDocsMutation()
2023/09/01 13:33:41 Index found:  index_age
2023/09/01 13:33:41 Using n1ql client
2023/09/01 13:33:41 Len of expected and actual scan results are :  10102 and 10102
2023/09/01 13:33:41 Expected and Actual scan responses are the same
2023/09/01 13:33:41 Using n1ql client
2023/09/01 13:33:41 Index Scan after mutations took 216.44997ms
2023/09/01 13:33:41 Len of expected and actual scan results are :  9902 and 9902
2023/09/01 13:33:41 Expected and Actual scan responses are the same
--- PASS: TestDeleteDocsMutation (0.64s)
=== RUN   TestUpdateDocsMutation
2023/09/01 13:33:41 In TestUpdateDocsMutation()
2023/09/01 13:33:41 Index found:  index_age
2023/09/01 13:33:41 Using n1ql client
2023/09/01 13:33:41 Len of expected and actual scan results are :  9439 and 9439
2023/09/01 13:33:41 Expected and Actual scan responses are the same
2023/09/01 13:33:41 Num of keysFromMutDocs: 100
2023/09/01 13:33:41 Updating number of documents: 100
2023/09/01 13:33:42 Using n1ql client
2023/09/01 13:33:42 Index Scan after mutations took 256.283144ms
2023/09/01 13:33:42 Len of expected and actual scan results are :  9435 and 9435
2023/09/01 13:33:42 Expected and Actual scan responses are the same
--- PASS: TestUpdateDocsMutation (0.57s)
=== RUN   TestLargeMutations
2023/09/01 13:33:42 In TestLargeMutations()
2023/09/01 13:33:42 In DropAllSecondaryIndexes()
2023/09/01 13:33:42 Index found:  index_isresidential
2023/09/01 13:33:42 Dropped index index_isresidential
2023/09/01 13:33:42 Index found:  index_streetaddress
2023/09/01 13:33:42 Dropped index index_streetaddress
2023/09/01 13:33:42 Index found:  index_ageabove30
2023/09/01 13:33:42 Dropped index index_ageabove30
2023/09/01 13:33:42 Index found:  index_nationality
2023/09/01 13:33:42 Dropped index index_nationality
2023/09/01 13:33:42 Index found:  index_cdc
2023/09/01 13:33:42 Dropped index index_cdc
2023/09/01 13:33:42 Index found:  index_streetname
2023/09/01 13:33:42 Dropped index index_streetname
2023/09/01 13:33:42 Index found:  idx_numbertest
2023/09/01 13:33:43 Dropped index idx_numbertest
2023/09/01 13:33:43 Index found:  index_floor
2023/09/01 13:33:43 Dropped index index_floor
2023/09/01 13:33:43 Index found:  index_eyeColor
2023/09/01 13:33:43 Dropped index index_eyeColor
2023/09/01 13:33:43 Index found:  index_company
2023/09/01 13:33:43 Dropped index index_company
2023/09/01 13:33:43 Index found:  idx_age
2023/09/01 13:33:43 Dropped index idx_age
2023/09/01 13:33:43 Index found:  index_tags
2023/09/01 13:33:43 Dropped index index_tags
2023/09/01 13:33:43 Index found:  index_composite2
2023/09/01 13:33:43 Dropped index index_composite2
2023/09/01 13:33:43 Index found:  index_pin
2023/09/01 13:33:43 Dropped index index_pin
2023/09/01 13:33:43 Index found:  index_gender
2023/09/01 13:33:43 Dropped index index_gender
2023/09/01 13:33:43 Index found:  index_isActive
2023/09/01 13:33:43 Dropped index index_isActive
2023/09/01 13:33:43 Index found:  index_deferred
2023/09/01 13:33:43 Dropped index index_deferred
2023/09/01 13:33:43 Index found:  index_di1
2023/09/01 13:33:44 Dropped index index_di1
2023/09/01 13:33:44 Index found:  index_p1
2023/09/01 13:33:44 Dropped index index_p1
2023/09/01 13:33:44 Index found:  index_sync_rev
2023/09/01 13:33:44 Dropped index index_sync_rev
2023/09/01 13:33:44 Index found:  index_i2
2023/09/01 13:33:44 Dropped index index_i2
2023/09/01 13:33:44 Index found:  index_name
2023/09/01 13:33:44 Dropped index index_name
2023/09/01 13:33:44 Index found:  index_composite1
2023/09/01 13:33:44 Dropped index index_composite1
2023/09/01 13:33:44 Index found:  index_email
2023/09/01 13:33:44 Dropped index index_email
2023/09/01 13:33:44 Index found:  index_age35to45
2023/09/01 13:33:44 Dropped index index_age35to45
2023/09/01 13:33:44 Index found:  index_balance
2023/09/01 13:33:44 Dropped index index_balance
2023/09/01 13:33:44 Index found:  index_sync_channels
2023/09/01 13:33:44 Dropped index index_sync_channels
2023/09/01 13:33:44 Index found:  index_longitude
2023/09/01 13:33:45 Dropped index index_longitude
2023/09/01 13:33:45 Index found:  index_age
2023/09/01 13:33:45 Dropped index index_age
2023/09/01 13:33:45 Index found:  index_latitude
2023/09/01 13:33:45 Dropped index index_latitude
2023/09/01 13:33:45 Index found:  index_state
2023/09/01 13:33:45 Dropped index index_state
2023/09/01 13:33:45 Index found:  index_height
2023/09/01 13:33:45 Dropped index index_height
2023/09/01 13:33:45 Index found:  index_ageteens
2023/09/01 13:33:45 Dropped index index_ageteens
2023/09/01 13:34:05 Created the secondary index indexmut_1. Waiting for it become active
2023/09/01 13:34:05 Index is 15267907952968185075 now active
2023/09/01 13:34:05 Using n1ql client
2023/09/01 13:34:05 Expected and Actual scan responses are the same
2023/09/01 13:34:05 Len of expected and actual scan results are :  29902 and 29902
2023/09/01 13:34:05 ITERATION 0
2023/09/01 13:34:23 Created the secondary index indexmut_2. Waiting for it become active
2023/09/01 13:34:23 Index is 14156464465678280952 now active
2023/09/01 13:34:23 Using n1ql client
2023/09/01 13:34:23 Expected and Actual scan responses are the same
2023/09/01 13:34:23 Len of expected and actual scan results are :  39902 and 39902
2023/09/01 13:34:23 Using n1ql client
2023/09/01 13:34:23 Expected and Actual scan responses are the same
2023/09/01 13:34:23 Len of expected and actual scan results are :  39902 and 39902
2023/09/01 13:34:23 Dropping the secondary index indexmut_2
2023/09/01 13:34:24 Index dropped
2023/09/01 13:34:24 ITERATION 1
2023/09/01 13:34:41 Created the secondary index indexmut_2. Waiting for it become active
2023/09/01 13:34:41 Index is 154766648974765003 now active
2023/09/01 13:34:41 Using n1ql client
2023/09/01 13:34:42 Expected and Actual scan responses are the same
2023/09/01 13:34:42 Len of expected and actual scan results are :  49902 and 49902
2023/09/01 13:34:42 Using n1ql client
2023/09/01 13:34:42 Expected and Actual scan responses are the same
2023/09/01 13:34:42 Len of expected and actual scan results are :  49902 and 49902
2023/09/01 13:34:42 Dropping the secondary index indexmut_2
2023/09/01 13:34:42 Index dropped
2023/09/01 13:34:42 ITERATION 2
2023/09/01 13:35:00 Created the secondary index indexmut_2. Waiting for it become active
2023/09/01 13:35:00 Index is 8456860919663141280 now active
2023/09/01 13:35:00 Using n1ql client
2023/09/01 13:35:01 Expected and Actual scan responses are the same
2023/09/01 13:35:01 Len of expected and actual scan results are :  59902 and 59902
2023/09/01 13:35:01 Using n1ql client
2023/09/01 13:35:01 Expected and Actual scan responses are the same
2023/09/01 13:35:01 Len of expected and actual scan results are :  59902 and 59902
2023/09/01 13:35:01 Dropping the secondary index indexmut_2
2023/09/01 13:35:01 Index dropped
2023/09/01 13:35:01 ITERATION 3
2023/09/01 13:35:20 Created the secondary index indexmut_2. Waiting for it become active
2023/09/01 13:35:20 Index is 16100935647119228995 now active
2023/09/01 13:35:20 Using n1ql client
2023/09/01 13:35:21 Expected and Actual scan responses are the same
2023/09/01 13:35:21 Len of expected and actual scan results are :  69902 and 69902
2023/09/01 13:35:21 Using n1ql client
2023/09/01 13:35:22 Expected and Actual scan responses are the same
2023/09/01 13:35:22 Len of expected and actual scan results are :  69902 and 69902
2023/09/01 13:35:22 Dropping the secondary index indexmut_2
2023/09/01 13:35:22 Index dropped
2023/09/01 13:35:22 ITERATION 4
2023/09/01 13:35:44 Created the secondary index indexmut_2. Waiting for it become active
2023/09/01 13:35:44 Index is 3661283364876715878 now active
2023/09/01 13:35:44 Using n1ql client
2023/09/01 13:35:44 Expected and Actual scan responses are the same
2023/09/01 13:35:44 Len of expected and actual scan results are :  79902 and 79902
2023/09/01 13:35:45 Using n1ql client
2023/09/01 13:35:45 Expected and Actual scan responses are the same
2023/09/01 13:35:45 Len of expected and actual scan results are :  79902 and 79902
2023/09/01 13:35:45 Dropping the secondary index indexmut_2
2023/09/01 13:35:45 Index dropped
2023/09/01 13:35:45 ITERATION 5
2023/09/01 13:36:05 Created the secondary index indexmut_2. Waiting for it become active
2023/09/01 13:36:05 Index is 8408427722283456302 now active
2023/09/01 13:36:05 Using n1ql client
2023/09/01 13:36:06 Expected and Actual scan responses are the same
2023/09/01 13:36:06 Len of expected and actual scan results are :  89902 and 89902
2023/09/01 13:36:06 Using n1ql client
2023/09/01 13:36:07 Expected and Actual scan responses are the same
2023/09/01 13:36:07 Len of expected and actual scan results are :  89902 and 89902
2023/09/01 13:36:07 Dropping the secondary index indexmut_2
2023/09/01 13:36:07 Index dropped
2023/09/01 13:36:07 ITERATION 6
2023/09/01 13:36:27 Created the secondary index indexmut_2. Waiting for it become active
2023/09/01 13:36:27 Index is 15207039770697280661 now active
2023/09/01 13:36:27 Using n1ql client
2023/09/01 13:36:28 Expected and Actual scan responses are the same
2023/09/01 13:36:28 Len of expected and actual scan results are :  99902 and 99902
2023/09/01 13:36:28 Using n1ql client
2023/09/01 13:36:28 Expected and Actual scan responses are the same
2023/09/01 13:36:28 Len of expected and actual scan results are :  99902 and 99902
2023/09/01 13:36:28 Dropping the secondary index indexmut_2
2023/09/01 13:36:28 Index dropped
2023/09/01 13:36:28 ITERATION 7
2023/09/01 13:36:50 Created the secondary index indexmut_2. Waiting for it become active
2023/09/01 13:36:50 Index is 829046970284343647 now active
2023/09/01 13:36:51 Using n1ql client
2023/09/01 13:36:52 Expected and Actual scan responses are the same
2023/09/01 13:36:52 Len of expected and actual scan results are :  109902 and 109902
2023/09/01 13:36:53 Using n1ql client
2023/09/01 13:36:54 Expected and Actual scan responses are the same
2023/09/01 13:36:54 Len of expected and actual scan results are :  109902 and 109902
2023/09/01 13:36:54 Dropping the secondary index indexmut_2
2023/09/01 13:36:54 Index dropped
2023/09/01 13:36:54 ITERATION 8
2023/09/01 13:37:17 Created the secondary index indexmut_2. Waiting for it become active
2023/09/01 13:37:17 Index is 8144683197142667295 now active
2023/09/01 13:37:17 Using n1ql client
2023/09/01 13:37:18 Expected and Actual scan responses are the same
2023/09/01 13:37:18 Len of expected and actual scan results are :  119902 and 119902
2023/09/01 13:37:18 Using n1ql client
2023/09/01 13:37:19 Expected and Actual scan responses are the same
2023/09/01 13:37:19 Len of expected and actual scan results are :  119902 and 119902
2023/09/01 13:37:19 Dropping the secondary index indexmut_2
2023/09/01 13:37:19 Index dropped
2023/09/01 13:37:19 ITERATION 9
2023/09/01 13:37:41 Created the secondary index indexmut_2. Waiting for it become active
2023/09/01 13:37:41 Index is 376342931129780204 now active
2023/09/01 13:37:42 Using n1ql client
2023/09/01 13:37:43 Expected and Actual scan responses are the same
2023/09/01 13:37:43 Len of expected and actual scan results are :  129902 and 129902
2023/09/01 13:37:43 Using n1ql client
2023/09/01 13:37:44 Expected and Actual scan responses are the same
2023/09/01 13:37:44 Len of expected and actual scan results are :  129902 and 129902
2023/09/01 13:37:44 Dropping the secondary index indexmut_2
2023/09/01 13:37:44 Index dropped
2023/09/01 13:37:44 ITERATION 10
2023/09/01 13:38:10 Created the secondary index indexmut_2. Waiting for it become active
2023/09/01 13:38:10 Index is 11239855480894516361 now active
2023/09/01 13:38:11 Using n1ql client
2023/09/01 13:38:12 Expected and Actual scan responses are the same
2023/09/01 13:38:12 Len of expected and actual scan results are :  139902 and 139902
2023/09/01 13:38:12 Using n1ql client
2023/09/01 13:38:13 Expected and Actual scan responses are the same
2023/09/01 13:38:13 Len of expected and actual scan results are :  139902 and 139902
2023/09/01 13:38:13 Dropping the secondary index indexmut_2
2023/09/01 13:38:13 Index dropped
--- PASS: TestLargeMutations (271.13s)
=== RUN   TestPlanner
2023/09/01 13:38:13 In TestPlanner()
2023/09/01 13:38:13 -------------------------------------------
2023/09/01 13:38:13 initial placement - 20-50M, 10 index, 3 replica, 2x
2023-09-01T13:38:13.554+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0
2023-09-01T13:38:13.561+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2023-09-01T13:38:13.561+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2023-09-01T13:38:13.563+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2023-09-01T13:38:13.563+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2023-09-01T13:38:13.567+05:30 [Info] switched currmeta from 481 -> 481 force true 
2023-09-01T13:38:13.569+05:30 [Info] switched currmeta from 475 -> 477 force true 
2023-09-01T13:38:13.593+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2023-09-01T13:38:13.593+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2023-09-01T13:38:13.595+05:30 [Info] switched currmeta from 477 -> 477 force true 
2023-09-01T13:38:13.596+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2023-09-01T13:38:13.596+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2023-09-01T13:38:13.598+05:30 [Info] switched currmeta from 481 -> 481 force true 
2023-09-01T13:38:13.661+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:13.661+05:30 [Info] Score: 0.044047384858279644
2023-09-01T13:38:13.661+05:30 [Info] Memory Quota: 57225507680 (53.2954G)
2023-09-01T13:38:13.661+05:30 [Info] CPU Quota: 12
2023-09-01T13:38:13.661+05:30 [Info] Indexer Memory Mean 34529545725 (32.1581G)
2023-09-01T13:38:13.661+05:30 [Info] Indexer Memory Deviation 3041872379 (2.83296G) (8.81%)
2023-09-01T13:38:13.661+05:30 [Info] Indexer Memory Utilization 0.6034
2023-09-01T13:38:13.661+05:30 [Info] Indexer CPU Mean 9.5936
2023-09-01T13:38:13.661+05:30 [Info] Indexer CPU Deviation 2.32 (24.14%)
2023-09-01T13:38:13.661+05:30 [Info] Indexer CPU Utilization 0.7995
2023-09-01T13:38:13.661+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:13.661+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:13.661+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:13.661+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:13.661+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:13.661+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:13.661+05:30 [Info] Indexer Data Size Mean 34529545725 (32.1581G)
2023-09-01T13:38:13.661+05:30 [Info] Indexer Data Size Deviation 3041872379 (2.83296G) (8.81%)
2023-09-01T13:38:13.661+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:13.661+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:13.661+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:13.661+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:13 -------------------------------------------
2023/09/01 13:38:13 initial placement - 20-50M, 30 index, 3 replica, 2x
2023-09-01T13:38:13.662+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0
2023-09-01T13:38:14.725+05:30 [Info] Score: 0.02642622777337
2023-09-01T13:38:14.725+05:30 [Info] Memory Quota: 64118530126 (59.715G)
2023-09-01T13:38:14.725+05:30 [Info] CPU Quota: 12
2023-09-01T13:38:14.725+05:30 [Info] Indexer Memory Mean 41430432891 (38.5851G)
2023-09-01T13:38:14.725+05:30 [Info] Indexer Memory Deviation 2189700112 (2.03932G) (5.29%)
2023-09-01T13:38:14.725+05:30 [Info] Indexer Memory Utilization 0.6462
2023-09-01T13:38:14.725+05:30 [Info] Indexer CPU Mean 11.0404
2023-09-01T13:38:14.725+05:30 [Info] Indexer CPU Deviation 2.47 (22.34%)
2023-09-01T13:38:14.725+05:30 [Info] Indexer CPU Utilization 0.9200
2023-09-01T13:38:14.725+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:14.725+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:14.725+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:14.725+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:14.725+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:14.725+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:14.725+05:30 [Info] Indexer Data Size Mean 41430432891 (38.5851G)
2023-09-01T13:38:14.725+05:30 [Info] Indexer Data Size Deviation 2189700112 (2.03932G) (5.29%)
2023-09-01T13:38:14.725+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:14.725+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:14.725+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:14.725+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:14 -------------------------------------------
2023/09/01 13:38:14 initial placement - 20-50M, 30 index, 3 replica, 4x
2023-09-01T13:38:14.726+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0
2023-09-01T13:38:14.984+05:30 [Info] Score: 0.01973218264635852
2023-09-01T13:38:14.984+05:30 [Info] Memory Quota: 127027067208 (118.303G)
2023-09-01T13:38:14.984+05:30 [Info] CPU Quota: 24
2023-09-01T13:38:14.984+05:30 [Info] Indexer Memory Mean 79466423084 (74.0089G)
2023-09-01T13:38:14.984+05:30 [Info] Indexer Memory Deviation 3136091949 (2.92071G) (3.95%)
2023-09-01T13:38:14.984+05:30 [Info] Indexer Memory Utilization 0.6256
2023-09-01T13:38:14.984+05:30 [Info] Indexer CPU Mean 21.5517
2023-09-01T13:38:14.984+05:30 [Info] Indexer CPU Deviation 3.65 (16.95%)
2023-09-01T13:38:14.984+05:30 [Info] Indexer CPU Utilization 0.8980
2023-09-01T13:38:14.984+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:14.984+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:14.984+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:14.984+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:14.984+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:14.984+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:14.984+05:30 [Info] Indexer Data Size Mean 79466423084 (74.0089G)
2023-09-01T13:38:14.984+05:30 [Info] Indexer Data Size Deviation 3136091949 (2.92071G) (3.95%)
2023-09-01T13:38:14.984+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:14.984+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:14.984+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:14.984+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:14 -------------------------------------------
2023/09/01 13:38:14 initial placement - 200-500M, 10 index, 3 replica, 2x
2023-09-01T13:38:14.985+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0
2023-09-01T13:38:15.020+05:30 [Info] Score: 0.031085689746460816
2023-09-01T13:38:15.020+05:30 [Info] Memory Quota: 559205315428 (520.801G)
2023-09-01T13:38:15.020+05:30 [Info] CPU Quota: 12
2023-09-01T13:38:15.020+05:30 [Info] Indexer Memory Mean 392861630140 (365.881G)
2023-09-01T13:38:15.020+05:30 [Info] Indexer Memory Deviation 24424749495 (22.7473G) (6.22%)
2023-09-01T13:38:15.020+05:30 [Info] Indexer Memory Utilization 0.7025
2023-09-01T13:38:15.020+05:30 [Info] Indexer CPU Mean 10.7878
2023-09-01T13:38:15.020+05:30 [Info] Indexer CPU Deviation 3.08 (28.54%)
2023-09-01T13:38:15.020+05:30 [Info] Indexer CPU Utilization 0.8990
2023-09-01T13:38:15.020+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:15.020+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:15.020+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:15.020+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:15.020+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:15.020+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:15.020+05:30 [Info] Indexer Data Size Mean 392861630140 (365.881G)
2023-09-01T13:38:15.020+05:30 [Info] Indexer Data Size Deviation 24424749495 (22.7473G) (6.22%)
2023-09-01T13:38:15.020+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:15.020+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:15.020+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:15.020+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:15 -------------------------------------------
2023/09/01 13:38:15 initial placement - 200-500M, 30 index, 3 replica, 2x
2023-09-01T13:38:15.021+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0
2023-09-01T13:38:16.029+05:30 [Info] Score: 0.03107850011740731
2023-09-01T13:38:16.029+05:30 [Info] Memory Quota: 529607732234 (493.236G)
2023-09-01T13:38:16.029+05:30 [Info] CPU Quota: 12
2023-09-01T13:38:16.029+05:30 [Info] Indexer Memory Mean 406973245064 (379.023G)
2023-09-01T13:38:16.029+05:30 [Info] Indexer Memory Deviation 25296236089 (23.559G) (6.22%)
2023-09-01T13:38:16.029+05:30 [Info] Indexer Memory Utilization 0.7684
2023-09-01T13:38:16.029+05:30 [Info] Indexer CPU Mean 9.6508
2023-09-01T13:38:16.029+05:30 [Info] Indexer CPU Deviation 2.80 (28.99%)
2023-09-01T13:38:16.029+05:30 [Info] Indexer CPU Utilization 0.8042
2023-09-01T13:38:16.029+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:16.029+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:16.029+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:16.029+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:16.029+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:16.029+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:16.029+05:30 [Info] Indexer Data Size Mean 406973245064 (379.023G)
2023-09-01T13:38:16.029+05:30 [Info] Indexer Data Size Deviation 25296236089 (23.559G) (6.22%)
2023-09-01T13:38:16.030+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:16.030+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:16.030+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:16.030+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:16 -------------------------------------------
2023/09/01 13:38:16 initial placement - mixed small/medium, 30 index, 3 replica, 1.5/4x
2023-09-01T13:38:16.030+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0
2023-09-01T13:38:16.330+05:30 [Info] Score: 0.0125245362956076
2023-09-01T13:38:16.330+05:30 [Info] Memory Quota: 348515528241 (324.58G)
2023-09-01T13:38:16.330+05:30 [Info] CPU Quota: 12
2023-09-01T13:38:16.330+05:30 [Info] Indexer Memory Mean 284929304150 (265.361G)
2023-09-01T13:38:16.330+05:30 [Info] Indexer Memory Deviation 7137214823 (6.64705G) (2.50%)
2023-09-01T13:38:16.330+05:30 [Info] Indexer Memory Utilization 0.8176
2023-09-01T13:38:16.330+05:30 [Info] Indexer CPU Mean 10.2960
2023-09-01T13:38:16.330+05:30 [Info] Indexer CPU Deviation 5.60 (54.43%)
2023-09-01T13:38:16.330+05:30 [Info] Indexer CPU Utilization 0.8580
2023-09-01T13:38:16.330+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:16.330+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:16.330+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:16.330+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:16.330+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:16.330+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:16.330+05:30 [Info] Indexer Data Size Mean 284929304150 (265.361G)
2023-09-01T13:38:16.330+05:30 [Info] Indexer Data Size Deviation 7137214823 (6.64705G) (2.50%)
2023-09-01T13:38:16.330+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:16.330+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:16.330+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:16.330+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:16 -------------------------------------------
2023/09/01 13:38:16 initial placement - mixed all, 30 index, 3 replica, 1.5/4x
2023-09-01T13:38:16.331+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0
2023-09-01T13:38:17.447+05:30 [Info] Score: 0.008951283030635982
2023-09-01T13:38:17.447+05:30 [Info] Memory Quota: 363477182031 (338.515G)
2023-09-01T13:38:17.447+05:30 [Info] CPU Quota: 20
2023-09-01T13:38:17.447+05:30 [Info] Indexer Memory Mean 296643378553 (276.271G)
2023-09-01T13:38:17.447+05:30 [Info] Indexer Memory Deviation 5310677681 (4.94595G) (1.79%)
2023-09-01T13:38:17.447+05:30 [Info] Indexer Memory Utilization 0.8161
2023-09-01T13:38:17.447+05:30 [Info] Indexer CPU Mean 8.2608
2023-09-01T13:38:17.447+05:30 [Info] Indexer CPU Deviation 3.84 (46.52%)
2023-09-01T13:38:17.447+05:30 [Info] Indexer CPU Utilization 0.4130
2023-09-01T13:38:17.447+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:17.447+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:17.447+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:17.447+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:17.447+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:17.447+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:17.447+05:30 [Info] Indexer Data Size Mean 296643378553 (276.271G)
2023-09-01T13:38:17.447+05:30 [Info] Indexer Data Size Deviation 5310677681 (4.94595G) (1.79%)
2023-09-01T13:38:17.447+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:17.447+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:17.448+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:17.448+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:17 -------------------------------------------
2023/09/01 13:38:17 initial placement - 6 2M index, 1 replica, 2x
2023-09-01T13:38:17.459+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0
2023-09-01T13:38:17.474+05:30 [Info] Score: 0
2023-09-01T13:38:17.475+05:30 [Info] Memory Quota: 4848128000 (4.51517G)
2023-09-01T13:38:17.475+05:30 [Info] CPU Quota: 2
2023-09-01T13:38:17.475+05:30 [Info] Indexer Memory Mean 2080000000 (1.93715G)
2023-09-01T13:38:17.475+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2023-09-01T13:38:17.475+05:30 [Info] Indexer Memory Utilization 0.4290
2023-09-01T13:38:17.475+05:30 [Info] Indexer CPU Mean 1.2000
2023-09-01T13:38:17.475+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2023-09-01T13:38:17.475+05:30 [Info] Indexer CPU Utilization 0.6000
2023-09-01T13:38:17.475+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:17.475+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:17.475+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:17.475+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:17.475+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:17.475+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:17.475+05:30 [Info] Indexer Data Size Mean 2080000000 (1.93715G)
2023-09-01T13:38:17.475+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2023-09-01T13:38:17.475+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:17.475+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:17.475+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:17.475+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:17 -------------------------------------------
2023/09/01 13:38:17 initial placement - 5 20M primary index, 2 replica, 2x
2023-09-01T13:38:17.485+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0
2023-09-01T13:38:17.526+05:30 [Info] Score: 0
2023-09-01T13:38:17.526+05:30 [Info] Memory Quota: 14310128000 (13.3273G)
2023-09-01T13:38:17.526+05:30 [Info] CPU Quota: 2
2023-09-01T13:38:17.526+05:30 [Info] Indexer Memory Mean 10960000000 (10.2073G)
2023-09-01T13:38:17.526+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2023-09-01T13:38:17.526+05:30 [Info] Indexer Memory Utilization 0.7659
2023-09-01T13:38:17.526+05:30 [Info] Indexer CPU Mean 1.2000
2023-09-01T13:38:17.526+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2023-09-01T13:38:17.526+05:30 [Info] Indexer CPU Utilization 0.6000
2023-09-01T13:38:17.526+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:17.526+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:17.526+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:17.526+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:17.526+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:17.526+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:17.526+05:30 [Info] Indexer Data Size Mean 10960000000 (10.2073G)
2023-09-01T13:38:17.526+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2023-09-01T13:38:17.526+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:17.526+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:17.526+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:17.526+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:17 -------------------------------------------
2023/09/01 13:38:17 initial placement - 5 20M array index, 2 replica, 2x
2023-09-01T13:38:17.536+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0
2023-09-01T13:38:17.576+05:30 [Info] Score: 0
2023-09-01T13:38:17.576+05:30 [Info] Memory Quota: 237416768000 (221.112G)
2023-09-01T13:38:17.576+05:30 [Info] CPU Quota: 2
2023-09-01T13:38:17.576+05:30 [Info] Indexer Memory Mean 191440000000 (178.292G)
2023-09-01T13:38:17.576+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2023-09-01T13:38:17.576+05:30 [Info] Indexer Memory Utilization 0.8063
2023-09-01T13:38:17.576+05:30 [Info] Indexer CPU Mean 1.2000
2023-09-01T13:38:17.576+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2023-09-01T13:38:17.576+05:30 [Info] Indexer CPU Utilization 0.6000
2023-09-01T13:38:17.576+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:17.576+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:17.576+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:17.576+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:17.577+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:17.577+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:17.577+05:30 [Info] Indexer Data Size Mean 191440000000 (178.292G)
2023-09-01T13:38:17.577+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2023-09-01T13:38:17.577+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:17.577+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:17.577+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:17.577+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:17 -------------------------------------------
2023/09/01 13:38:17 initial placement - 3 replica constraint, 2 index, 2x
2023-09-01T13:38:17.580+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0
2023-09-01T13:38:17.621+05:30 [Info] Score: 0
2023-09-01T13:38:17.621+05:30 [Info] Memory Quota: 530294000 (505.728M)
2023-09-01T13:38:17.621+05:30 [Info] CPU Quota: 2
2023-09-01T13:38:17.621+05:30 [Info] Indexer Memory Mean 2600000 (2.47955M)
2023-09-01T13:38:17.621+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2023-09-01T13:38:17.621+05:30 [Info] Indexer Memory Utilization 0.0049
2023-09-01T13:38:17.621+05:30 [Info] Indexer CPU Mean 0.0000
2023-09-01T13:38:17.621+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2023-09-01T13:38:17.621+05:30 [Info] Indexer CPU Utilization 0.0000
2023-09-01T13:38:17.621+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:17.621+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:17.621+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:17.621+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:17.621+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:17.621+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:17.621+05:30 [Info] Indexer Data Size Mean 2600000 (2.47955M)
2023-09-01T13:38:17.621+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2023-09-01T13:38:17.621+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:17.621+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:17.621+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:17.621+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:17 -------------------------------------------
2023/09/01 13:38:17 incr placement - 20-50M, 5 2M index, 1 replica, 1x
2023-09-01T13:38:17.631+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.027603093206444586
2023-09-01T13:38:17.776+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:17.777+05:30 [Info] Score: 0.007189938568471697
2023-09-01T13:38:17.777+05:30 [Info] Memory Quota: 125233041042 (116.632G)
2023-09-01T13:38:17.777+05:30 [Info] CPU Quota: 27
2023-09-01T13:38:17.777+05:30 [Info] Indexer Memory Mean 71117238485 (66.2331G)
2023-09-01T13:38:17.777+05:30 [Info] Indexer Memory Deviation 1022657151 (975.282M) (1.44%)
2023-09-01T13:38:17.777+05:30 [Info] Indexer Memory Utilization 0.5679
2023-09-01T13:38:17.777+05:30 [Info] Indexer CPU Mean 20.1734
2023-09-01T13:38:17.777+05:30 [Info] Indexer CPU Deviation 1.87 (9.29%)
2023-09-01T13:38:17.777+05:30 [Info] Indexer CPU Utilization 0.7472
2023-09-01T13:38:17.777+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:17.777+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:17.777+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:17.777+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:17.777+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:17.777+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:17.777+05:30 [Info] Indexer Data Size Mean 71117238485 (66.2331G)
2023-09-01T13:38:17.777+05:30 [Info] Indexer Data Size Deviation 1022657151 (975.282M) (1.44%)
2023-09-01T13:38:17.777+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:17.777+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:17.777+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:17.777+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:17 -------------------------------------------
2023/09/01 13:38:17 incr placement - mixed small/medium, 6 2M index, 1 replica, 1x
2023-09-01T13:38:17.789+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.0019696306157357887
2023-09-01T13:38:18.034+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:18.034+05:30 [Info] Score: 0.00043718596495524715
2023-09-01T13:38:18.034+05:30 [Info] Memory Quota: 536870912000 (500G)
2023-09-01T13:38:18.034+05:30 [Info] CPU Quota: 20
2023-09-01T13:38:18.034+05:30 [Info] Indexer Memory Mean 393025602195 (366.034G)
2023-09-01T13:38:18.034+05:30 [Info] Indexer Memory Deviation 343650554 (327.731M) (0.09%)
2023-09-01T13:38:18.034+05:30 [Info] Indexer Memory Utilization 0.7321
2023-09-01T13:38:18.034+05:30 [Info] Indexer CPU Mean 14.2305
2023-09-01T13:38:18.034+05:30 [Info] Indexer CPU Deviation 0.95 (6.65%)
2023-09-01T13:38:18.034+05:30 [Info] Indexer CPU Utilization 0.7115
2023-09-01T13:38:18.034+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:18.034+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:18.034+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:18.034+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:18.034+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:18.034+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:18.034+05:30 [Info] Indexer Data Size Mean 393025602195 (366.034G)
2023-09-01T13:38:18.035+05:30 [Info] Indexer Data Size Deviation 343650554 (327.731M) (0.09%)
2023-09-01T13:38:18.035+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:18.035+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:18.035+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:18.035+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:18 -------------------------------------------
2023/09/01 13:38:18 incr placement - 3 server group, 3 replica, 1x
2023-09-01T13:38:18.038+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.816496580927726
2023-09-01T13:38:18.064+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:18.064+05:30 [Info] Score: 0
2023-09-01T13:38:18.064+05:30 [Info] Memory Quota: 530294000 (505.728M)
2023-09-01T13:38:18.064+05:30 [Info] CPU Quota: 16
2023-09-01T13:38:18.064+05:30 [Info] Indexer Memory Mean 2600000 (2.47955M)
2023-09-01T13:38:18.064+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2023-09-01T13:38:18.064+05:30 [Info] Indexer Memory Utilization 0.0049
2023-09-01T13:38:18.064+05:30 [Info] Indexer CPU Mean 0.0000
2023-09-01T13:38:18.064+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2023-09-01T13:38:18.064+05:30 [Info] Indexer CPU Utilization 0.0000
2023-09-01T13:38:18.064+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:18.064+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:18.064+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:18.064+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:18.064+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:18.064+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:18.064+05:30 [Info] Indexer Data Size Mean 2600000 (2.47955M)
2023-09-01T13:38:18.065+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2023-09-01T13:38:18.065+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:18.065+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:18.065+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:18.065+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:18 -------------------------------------------
2023/09/01 13:38:18 incr placement - 2 server group, 3 replica, 1x
2023-09-01T13:38:18.067+05:30 [Warn] Index has more replica than server group. Index=index1 0 Bucket=bucket2 Scope= Collection=
2023-09-01T13:38:18.067+05:30 [Warn] Index has more replica than server group. Index=index1 0 (replica 1) Bucket=bucket2 Scope= Collection=
2023-09-01T13:38:18.067+05:30 [Warn] Index has more replica than server group. Index=index1 0 (replica 2) Bucket=bucket2 Scope= Collection=
2023-09-01T13:38:18.067+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0
2023-09-01T13:38:18.081+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:18.081+05:30 [Info] Score: 0
2023-09-01T13:38:18.081+05:30 [Info] Memory Quota: 530294000 (505.728M)
2023-09-01T13:38:18.081+05:30 [Info] CPU Quota: 16
2023-09-01T13:38:18.081+05:30 [Info] Indexer Memory Mean 2600000 (2.47955M)
2023-09-01T13:38:18.081+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2023-09-01T13:38:18.081+05:30 [Info] Indexer Memory Utilization 0.0049
2023-09-01T13:38:18.081+05:30 [Info] Indexer CPU Mean 0.0000
2023-09-01T13:38:18.081+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2023-09-01T13:38:18.081+05:30 [Info] Indexer CPU Utilization 0.0000
2023-09-01T13:38:18.081+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:18.081+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:18.081+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:18.081+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:18.081+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:18.081+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:18.081+05:30 [Info] Indexer Data Size Mean 2600000 (2.47955M)
2023-09-01T13:38:18.081+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2023-09-01T13:38:18.081+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:18.081+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:18.081+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:18.081+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:18 -------------------------------------------
2023/09/01 13:38:18 rebalance - 20-50M, 90 index, 20% shuffle, 1x, utilization 90%+
2023-09-01T13:38:18.083+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.22966764242275323
2023-09-01T13:38:18.083+05:30 [Info] Planner::initial temperature: initial resource variation 0.22966764242275323 temp 0.022966764242275325
2023-09-01T13:38:18.262+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:18.262+05:30 [Info] Score: 0.21074332296345175
2023-09-01T13:38:18.262+05:30 [Info] Memory Quota: 139586437120 (130G)
2023-09-01T13:38:18.262+05:30 [Info] CPU Quota: 30
2023-09-01T13:38:18.262+05:30 [Info] Indexer Memory Mean 88876568718 (82.7728G)
2023-09-01T13:38:18.262+05:30 [Info] Indexer Memory Deviation 3994277605 (3.71996G) (4.49%)
2023-09-01T13:38:18.262+05:30 [Info] Indexer Memory Utilization 0.6367
2023-09-01T13:38:18.262+05:30 [Info] Indexer CPU Mean 24.0538
2023-09-01T13:38:18.262+05:30 [Info] Indexer CPU Deviation 3.83 (15.93%)
2023-09-01T13:38:18.262+05:30 [Info] Indexer CPU Utilization 0.8018
2023-09-01T13:38:18.262+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:18.262+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:18.262+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:18.262+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:18.262+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:18.262+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:18.262+05:30 [Info] Indexer Data Size Mean 88876568718 (82.7728G)
2023-09-01T13:38:18.262+05:30 [Info] Indexer Data Size Deviation 3994277605 (3.71996G) (4.49%)
2023-09-01T13:38:18.262+05:30 [Info] Total Index Data (from non-deleted node) 993.273G
2023-09-01T13:38:18.262+05:30 [Info] Index Data Moved (exclude new node) 419.023G (42.19%)
2023-09-01T13:38:18.262+05:30 [Info] No. Index (from non-deleted node) 90
2023-09-01T13:38:18.262+05:30 [Info] No. Index Moved (exclude new node) 33 (36.67%)
2023/09/01 13:38:18 -------------------------------------------
2023/09/01 13:38:18 rebalance - mixed small/medium, 90 index, 20% shuffle, 1x
2023-09-01T13:38:18.264+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.3328942779069544
2023-09-01T13:38:18.264+05:30 [Info] Planner::initial temperature: initial resource variation 0.3328942779069544 temp 0.03328942779069544
2023-09-01T13:38:18.270+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification
2023-09-01T13:38:18.275+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2023-09-01T13:38:18.275+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2023-09-01T13:38:18.276+05:30 [Info] switched currmeta from 481 -> 481 force true 
2023-09-01T13:38:18.277+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh.
2023-09-01T13:38:18.277+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106]
2023-09-01T13:38:18.279+05:30 [Info] switched currmeta from 477 -> 477 force true 
2023-09-01T13:38:18.499+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:18.499+05:30 [Info] Score: 0.20088587735480634
2023-09-01T13:38:18.499+05:30 [Info] Memory Quota: 536870912000 (500G)
2023-09-01T13:38:18.499+05:30 [Info] CPU Quota: 20
2023-09-01T13:38:18.500+05:30 [Info] Indexer Memory Mean 392505602195 (365.549G)
2023-09-01T13:38:18.500+05:30 [Info] Indexer Memory Deviation 29090903807 (27.093G) (7.41%)
2023-09-01T13:38:18.500+05:30 [Info] Indexer Memory Utilization 0.7311
2023-09-01T13:38:18.500+05:30 [Info] Indexer CPU Mean 13.9305
2023-09-01T13:38:18.500+05:30 [Info] Indexer CPU Deviation 2.35 (16.85%)
2023-09-01T13:38:18.500+05:30 [Info] Indexer CPU Utilization 0.6965
2023-09-01T13:38:18.500+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:18.500+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:18.500+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:18.500+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:18.500+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:18.500+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:18.500+05:30 [Info] Indexer Data Size Mean 392505602195 (365.549G)
2023-09-01T13:38:18.500+05:30 [Info] Indexer Data Size Deviation 29090903807 (27.093G) (7.41%)
2023-09-01T13:38:18.500+05:30 [Info] Total Index Data (from non-deleted node) 4.28378T
2023-09-01T13:38:18.500+05:30 [Info] Index Data Moved (exclude new node) 2.03238T (47.44%)
2023-09-01T13:38:18.500+05:30 [Info] No. Index (from non-deleted node) 90
2023-09-01T13:38:18.500+05:30 [Info] No. Index Moved (exclude new node) 21 (23.33%)
2023/09/01 13:38:18 -------------------------------------------
2023/09/01 13:38:18 rebalance - travel sample, 10% shuffle, 1x
2023-09-01T13:38:18.501+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.08263955061545106
2023-09-01T13:38:18.501+05:30 [Info] Planner::initial temperature: initial resource variation 0.08263955061545106 temp 0.008263955061545106
2023-09-01T13:38:18.516+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:18.516+05:30 [Info] Score: 0.020659887653862764
2023-09-01T13:38:18.516+05:30 [Info] Memory Quota: 536870912 (512M)
2023-09-01T13:38:18.516+05:30 [Info] CPU Quota: 8
2023-09-01T13:38:18.516+05:30 [Info] Indexer Memory Mean 17503138 (16.6923M)
2023-09-01T13:38:18.516+05:30 [Info] Indexer Memory Deviation 1446451 (1.37944M) (8.26%)
2023-09-01T13:38:18.516+05:30 [Info] Indexer Memory Utilization 0.0326
2023-09-01T13:38:18.516+05:30 [Info] Indexer CPU Mean 0.0000
2023-09-01T13:38:18.516+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2023-09-01T13:38:18.516+05:30 [Info] Indexer CPU Utilization 0.0000
2023-09-01T13:38:18.516+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:18.516+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:18.516+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:18.516+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:18.516+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:18.516+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:18.516+05:30 [Info] Indexer Data Size Mean 0 (0)
2023-09-01T13:38:18.516+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2023-09-01T13:38:18.516+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:18.516+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:18.516+05:30 [Info] No. Index (from non-deleted node) 10
2023-09-01T13:38:18.516+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:18 -------------------------------------------
2023/09/01 13:38:18 rebalance - 20-50M, 90 index, swap 2, 1x
2023-09-01T13:38:18.518+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 2596996162
2023-09-01T13:38:18.518+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 4039455774
2023-09-01T13:38:18.518+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 2854263694
2023-09-01T13:38:18.518+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 1879968118
2023-09-01T13:38:18.518+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 1823804162
2023-09-01T13:38:18.518+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 2949882636
2023-09-01T13:38:18.518+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 281908850
2023-09-01T13:38:18.518+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 672245080
2023-09-01T13:38:18.518+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 416480912
2023-09-01T13:38:18.518+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 1292406600
2023-09-01T13:38:18.518+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 2212821389
2023-09-01T13:38:18.518+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 3494557023
2023-09-01T13:38:18.518+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.4095843409677649
2023-09-01T13:38:18.518+05:30 [Info] Planner::initial temperature: initial resource variation 0.4095843409677649 temp 0.04095843409677649
2023-09-01T13:38:18.746+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:18.746+05:30 [Info] Score: 0.014867597223215703
2023-09-01T13:38:18.746+05:30 [Info] Memory Quota: 139586437120 (130G)
2023-09-01T13:38:18.746+05:30 [Info] CPU Quota: 30
2023-09-01T13:38:18.746+05:30 [Info] Indexer Memory Mean 88876568718 (82.7728G)
2023-09-01T13:38:18.746+05:30 [Info] Indexer Memory Deviation 2642762052 (2.46126G) (2.97%)
2023-09-01T13:38:18.746+05:30 [Info] Indexer Memory Utilization 0.6367
2023-09-01T13:38:18.746+05:30 [Info] Indexer CPU Mean 24.0538
2023-09-01T13:38:18.746+05:30 [Info] Indexer CPU Deviation 0.65 (2.70%)
2023-09-01T13:38:18.746+05:30 [Info] Indexer CPU Utilization 0.8018
2023-09-01T13:38:18.746+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:18.746+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:18.746+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:18.746+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:18.746+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:18.746+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:18.746+05:30 [Info] Indexer Data Size Mean 88876568718 (82.7728G)
2023-09-01T13:38:18.746+05:30 [Info] Indexer Data Size Deviation 2642762052 (2.46126G) (2.97%)
2023-09-01T13:38:18.746+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:18.746+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:18.746+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:18.746+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:18 -------------------------------------------
2023/09/01 13:38:18 rebalance - mixed small/medium, 90 index, swap 2, 1x
2023-09-01T13:38:18.748+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 2596996162
2023-09-01T13:38:18.748+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 4039455774
2023-09-01T13:38:18.748+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 2854263694
2023-09-01T13:38:18.748+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 1879968118
2023-09-01T13:38:18.748+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 1823804162
2023-09-01T13:38:18.748+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 2949882636
2023-09-01T13:38:18.748+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 281908850
2023-09-01T13:38:18.748+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 672245080
2023-09-01T13:38:18.748+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 416480912
2023-09-01T13:38:18.748+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 1292406600
2023-09-01T13:38:18.748+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 2212821389
2023-09-01T13:38:18.748+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 3494557023
2023-09-01T13:38:18.748+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.4082556623939647
2023-09-01T13:38:18.748+05:30 [Info] Planner::initial temperature: initial resource variation 0.4082556623939647 temp 0.040825566239396475
2023-09-01T13:38:18.951+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:18.951+05:30 [Info] Score: 0.0011199297161658313
2023-09-01T13:38:18.951+05:30 [Info] Memory Quota: 536870912000 (500G)
2023-09-01T13:38:18.951+05:30 [Info] CPU Quota: 20
2023-09-01T13:38:18.951+05:30 [Info] Indexer Memory Mean 392505602195 (365.549G)
2023-09-01T13:38:18.951+05:30 [Info] Indexer Memory Deviation 879157375 (838.43M) (0.22%)
2023-09-01T13:38:18.951+05:30 [Info] Indexer Memory Utilization 0.7311
2023-09-01T13:38:18.951+05:30 [Info] Indexer CPU Mean 13.9305
2023-09-01T13:38:18.951+05:30 [Info] Indexer CPU Deviation 0.98 (7.04%)
2023-09-01T13:38:18.951+05:30 [Info] Indexer CPU Utilization 0.6965
2023-09-01T13:38:18.951+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:18.952+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:18.952+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:18.952+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:18.952+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:18.952+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:18.952+05:30 [Info] Indexer Data Size Mean 392505602195 (365.549G)
2023-09-01T13:38:18.952+05:30 [Info] Indexer Data Size Deviation 879157375 (838.43M) (0.22%)
2023-09-01T13:38:18.952+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:18.952+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:18.952+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:18.952+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:18 -------------------------------------------
2023/09/01 13:38:18 rebalance - travel sample, swap 2, 1x
2023-09-01T13:38:18.952+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 192.168.1.8:9000
2023-09-01T13:38:18.952+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 127.0.0.1:9001
2023-09-01T13:38:18.952+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 1.0086622738872069
2023-09-01T13:38:18.952+05:30 [Info] Planner::initial temperature: initial resource variation 1.0086622738872069 temp 0.1008662273887207
2023-09-01T13:38:19.088+05:30 [Info] Score: 0.0003181929343700274
2023-09-01T13:38:19.088+05:30 [Info] Memory Quota: 536870912 (512M)
2023-09-01T13:38:19.088+05:30 [Info] CPU Quota: 8
2023-09-01T13:38:19.088+05:30 [Info] Indexer Memory Mean 17503138 (16.6923M)
2023-09-01T13:38:19.088+05:30 [Info] Indexer Memory Deviation 22277 (21.7549K) (0.13%)
2023-09-01T13:38:19.088+05:30 [Info] Indexer Memory Utilization 0.0326
2023-09-01T13:38:19.088+05:30 [Info] Indexer CPU Mean 0.0000
2023-09-01T13:38:19.088+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2023-09-01T13:38:19.088+05:30 [Info] Indexer CPU Utilization 0.0000
2023-09-01T13:38:19.088+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:19.088+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:19.088+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:19.088+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:19.088+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:19.088+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:19.088+05:30 [Info] Indexer Data Size Mean 0 (0)
2023-09-01T13:38:19.088+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2023-09-01T13:38:19.088+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:19.088+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:19.088+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:19.088+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:19 -------------------------------------------
2023/09/01 13:38:19 rebalance - 8 identical index, add 4, 1x
2023-09-01T13:38:19.089+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 2596996162
2023-09-01T13:38:19.089+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 4039455774
2023-09-01T13:38:19.089+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 2854263694
2023-09-01T13:38:19.089+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 1879968118
2023-09-01T13:38:19.089+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 1
2023-09-01T13:38:19.089+05:30 [Info] Planner::initial temperature: initial resource variation 1 temp 0.1
2023-09-01T13:38:21.148+05:30 [Info] connected with 1 indexers
2023-09-01T13:38:21.148+05:30 [Info] client stats current counts: current: 1, not current: 0
2023-09-01T13:38:21.161+05:30 [Info] num concurrent scans {0}
2023-09-01T13:38:21.161+05:30 [Info] average scan response {1 ms}
2023-09-01T13:38:21.894+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2023-09-01T13:38:22.349+05:30 [Info] connected with 1 indexers
2023-09-01T13:38:22.349+05:30 [Info] client stats current counts: current: 1, not current: 0
2023-09-01T13:38:22.355+05:30 [Info] num concurrent scans {0}
2023-09-01T13:38:22.355+05:30 [Info] average scan response {556 ms}
2023-09-01T13:38:22.355+05:30 [Info] GSIC[default/default-_default-_default-1693554802307773842] logstats "default" {"gsi_scan_count":109,"gsi_scan_duration":11647483117,"gsi_throttle_duration":1069534992,"gsi_prime_duration":1149677979,"gsi_blocked_duration":1414111574,"gsi_total_temp_files":0,"gsi_backfill_size":0}
2023-09-01T13:38:23.025+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1
2023-09-01T13:38:23.100+05:30 [Info] Score: 0
2023-09-01T13:38:23.100+05:30 [Info] Memory Quota: 930294000 (887.197M)
2023-09-01T13:38:23.100+05:30 [Info] CPU Quota: 2
2023-09-01T13:38:23.100+05:30 [Info] Indexer Memory Mean 260000000 (247.955M)
2023-09-01T13:38:23.100+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2023-09-01T13:38:23.100+05:30 [Info] Indexer Memory Utilization 0.2795
2023-09-01T13:38:23.100+05:30 [Info] Indexer CPU Mean 0.0000
2023-09-01T13:38:23.100+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2023-09-01T13:38:23.100+05:30 [Info] Indexer CPU Utilization 0.0000
2023-09-01T13:38:23.100+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:23.100+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:23.100+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:23.100+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:23.100+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:23.100+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:23.100+05:30 [Info] Indexer Data Size Mean 260000000 (247.955M)
2023-09-01T13:38:23.100+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2023-09-01T13:38:23.100+05:30 [Info] Total Index Data (from non-deleted node) 1.93715G
2023-09-01T13:38:23.100+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:23.100+05:30 [Info] No. Index (from non-deleted node) 8
2023-09-01T13:38:23.100+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:23 -------------------------------------------
2023/09/01 13:38:23 rebalance - 8 identical index, delete 2, 2x
2023-09-01T13:38:23.101+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0
2023-09-01T13:38:23.101+05:30 [Info] Planner::initial temperature: initial resource variation 0 temp 1e-05
2023-09-01T13:38:23.149+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:23.149+05:30 [Info] Score: 0
2023-09-01T13:38:23.149+05:30 [Info] Memory Quota: 1060588000 (1011.46M)
2023-09-01T13:38:23.149+05:30 [Info] CPU Quota: 4
2023-09-01T13:38:23.149+05:30 [Info] Indexer Memory Mean 10400000 (9.91821M)
2023-09-01T13:38:23.149+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2023-09-01T13:38:23.149+05:30 [Info] Indexer Memory Utilization 0.0098
2023-09-01T13:38:23.149+05:30 [Info] Indexer CPU Mean 0.0000
2023-09-01T13:38:23.149+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2023-09-01T13:38:23.149+05:30 [Info] Indexer CPU Utilization 0.0000
2023-09-01T13:38:23.149+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:23.149+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:23.149+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:23.149+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:23.149+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:23.149+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:23.149+05:30 [Info] Indexer Data Size Mean 10400000 (9.91821M)
2023-09-01T13:38:23.149+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2023-09-01T13:38:23.149+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:23.149+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:23.149+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:23.149+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:23 -------------------------------------------
2023/09/01 13:38:23 rebalance - drop replcia - 3 replica, 3 zone, delete 1, 2x
2023-09-01T13:38:23.150+05:30 [Warn] There is more replica than available nodes.  Will not move index replica (default,,,country) from ejected node 127.0.0.1:9003
2023-09-01T13:38:23.150+05:30 [Info] Planner::planSingleRun Initial variance of the solution: NaN
2023-09-01T13:38:23.150+05:30 [Info] Planner::initial temperature: initial resource variation NaN temp NaN
2023-09-01T13:38:23.150+05:30 [Info] Score: 0
2023-09-01T13:38:23.150+05:30 [Info] Memory Quota: 536870912 (512M)
2023-09-01T13:38:23.150+05:30 [Info] CPU Quota: 16
2023-09-01T13:38:23.150+05:30 [Info] Indexer Memory Mean 0 (0)
2023-09-01T13:38:23.150+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2023-09-01T13:38:23.150+05:30 [Info] Indexer Memory Utilization 0.0000
2023-09-01T13:38:23.150+05:30 [Info] Indexer CPU Mean 0.0000
2023-09-01T13:38:23.150+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2023-09-01T13:38:23.150+05:30 [Info] Indexer CPU Utilization 0.0000
2023-09-01T13:38:23.150+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:23.150+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:23.150+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:23.150+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:23.150+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:23.150+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:23.151+05:30 [Info] Indexer Data Size Mean 0 (0)
2023-09-01T13:38:23.151+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2023-09-01T13:38:23.151+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:23.151+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:23.151+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:23.151+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:23 -------------------------------------------
2023/09/01 13:38:23 rebalance - rebuid replica - 3 replica, 3 zone, add 1, delete 1, 1x
2023-09-01T13:38:23.151+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 127.0.0.1:9001
2023-09-01T13:38:23.151+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 127.0.0.1:9002
2023-09-01T13:38:23.151+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 127.0.0.1:9003
2023-09-01T13:38:23.151+05:30 [Info] Planner::planSingleRun Initial variance of the solution: NaN
2023-09-01T13:38:23.151+05:30 [Info] Planner::initial temperature: initial resource variation NaN temp NaN
2023-09-01T13:38:23.151+05:30 [Info] Planner::finalizing the solution as final solution is found.
2023-09-01T13:38:23.151+05:30 [Info] Score: 0
2023-09-01T13:38:23.151+05:30 [Info] Memory Quota: 268435456 (256M)
2023-09-01T13:38:23.151+05:30 [Info] CPU Quota: 8
2023-09-01T13:38:23.151+05:30 [Info] Indexer Memory Mean 0 (0)
2023-09-01T13:38:23.151+05:30 [Info] Indexer Memory Deviation 0 (0) (0.00%)
2023-09-01T13:38:23.151+05:30 [Info] Indexer Memory Utilization 0.0000
2023-09-01T13:38:23.151+05:30 [Info] Indexer CPU Mean 0.0000
2023-09-01T13:38:23.151+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2023-09-01T13:38:23.151+05:30 [Info] Indexer CPU Utilization 0.0000
2023-09-01T13:38:23.151+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:23.151+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:23.151+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:23.151+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:23.151+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:23.151+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:23.151+05:30 [Info] Indexer Data Size Mean 0 (0)
2023-09-01T13:38:23.151+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2023-09-01T13:38:23.151+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:23.151+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:23.151+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:23.151+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023/09/01 13:38:23 -------------------------------------------
2023/09/01 13:38:23 Minimum memory test 1: min memory = 0
2023-09-01T13:38:23.152+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.25
2023-09-01T13:38:23.152+05:30 [Info] Planner::initial temperature: initial resource variation 0.25 temp 0.025
2023-09-01T13:38:23.154+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023/09/01 13:38:23 -------------------------------------------
2023/09/01 13:38:23 Minimum memory test 2: min memory > quota
2023-09-01T13:38:23.155+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=1.  Elapsed Time=5us
2023-09-01T13:38:23.155+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=2.  Elapsed Time=5us
2023-09-01T13:38:23.155+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=3.  Elapsed Time=3us
2023-09-01T13:38:23.155+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.25
2023-09-01T13:38:23.155+05:30 [Info] Planner::initial temperature: initial resource variation 0.25 temp 0.025
2023/09/01 13:38:24 -------------------------------------------
2023/09/01 13:38:24 Minimum memory test 3: min memory < quota
2023-09-01T13:38:24.210+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.25
2023-09-01T13:38:24.210+05:30 [Info] Planner::initial temperature: initial resource variation 0.25 temp 0.025
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Minimum memory test 4: replica repair with min memory > quota
2023-09-01T13:38:25.199+05:30 [Info] Rebuilding lost replica for (default,,,country,0,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.199+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=1.  Elapsed Time=6us
2023-09-01T13:38:25.199+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=2.  Elapsed Time=5us
2023-09-01T13:38:25.199+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=3.  Elapsed Time=4us
2023-09-01T13:38:25.199+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.3535533905932738
2023-09-01T13:38:25.199+05:30 [Info] Planner::initial temperature: initial resource variation 0.3535533905932738 temp 0.03535533905932738
2023-09-01T13:38:25.199+05:30 [Info] Planner::finalizing the solution as final solution is found.
2023-09-01T13:38:25.199+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:25.200+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=4.  Elapsed Time=265us
2023-09-01T13:38:25.200+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.3535533905932738
2023-09-01T13:38:25.200+05:30 [Info] Planner::initial temperature: initial resource variation 0.3535533905932738 temp 0.03535533905932738
2023-09-01T13:38:25.200+05:30 [Info] Planner::finalizing the solution as final solution is found.
2023-09-01T13:38:25.200+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:25.200+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=5.  Elapsed Time=199us
2023-09-01T13:38:25.200+05:30 [Info] Cannot rebuild lost replica due to resource constraint in cluster.  Will not rebuild lost replica.
2023-09-01T13:38:25.200+05:30 [Warn] 
MemoryQuota: 200
CpuQuota: 8
--- Violations for index  (mem 130, cpu 0) at node 127.0.0.1:9003 
	Cannot move to 127.0.0.1:9001: ReplicaViolation (free mem 1.67772e+07T, free cpu 8)
	Cannot move to 127.0.0.1:9002: ReplicaViolation (free mem 1.67772e+07T, free cpu 8)
2023-09-01T13:38:25.200+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0
2023-09-01T13:38:25.200+05:30 [Info] Planner::planSingleRun Skip running planner as current solution resource variation: 0 is less than threshold: 0. No nodes have been added or deleted and there are no violations observed
2023-09-01T13:38:25.200+05:30 [Info] Planner::initial temperature: initial resource variation 0 temp 1e-05
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Minimum memory test 5: replica repair with min memory < quota
2023-09-01T13:38:25.200+05:30 [Info] Rebuilding lost replica for (default,,,country,0,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.200+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.3535533905932738
2023-09-01T13:38:25.200+05:30 [Info] Planner::initial temperature: initial resource variation 0.3535533905932738 temp 0.03535533905932738
2023-09-01T13:38:25.206+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Minimum memory test 6: rebalance with min memory > quota
2023-09-01T13:38:25.207+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=1.  Elapsed Time=5us
2023-09-01T13:38:25.207+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=2.  Elapsed Time=7us
2023-09-01T13:38:25.207+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=3.  Elapsed Time=7us
2023-09-01T13:38:25.207+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=4.  Elapsed Time=6us
2023-09-01T13:38:25.207+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=5.  Elapsed Time=6us
2023-09-01T13:38:25.207+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=6.  Elapsed Time=6us
2023-09-01T13:38:25.207+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=7.  Elapsed Time=5us
2023-09-01T13:38:25.207+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=8.  Elapsed Time=7us
2023-09-01T13:38:25.207+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=9.  Elapsed Time=6us
2023-09-01T13:38:25.207+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=10.  Elapsed Time=12us
2023-09-01T13:38:25.207+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=11.  Elapsed Time=6us
2023-09-01T13:38:25.207+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=12.  Elapsed Time=6us
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Minimum memory test 7: rebalance-out with min memory > quota
2023-09-01T13:38:25.207+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=1.  Elapsed Time=5us
2023-09-01T13:38:25.207+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=2.  Elapsed Time=6us
2023-09-01T13:38:25.207+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=3.  Elapsed Time=7us
2023-09-01T13:38:25.207+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=4.  Elapsed Time=6us
2023-09-01T13:38:25.207+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=5.  Elapsed Time=5us
2023-09-01T13:38:25.207+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=6.  Elapsed Time=5us
2023-09-01T13:38:25.207+05:30 [Warn] Unable to find a solution with resource constraint.  Relax resource constraint check.
2023-09-01T13:38:25.208+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.25
2023-09-01T13:38:25.208+05:30 [Info] Planner::initial temperature: initial resource variation 0.25 temp 0.025
2023-09-01T13:38:25.208+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Minimum memory test 8: plan with min memory > quota
2023-09-01T13:38:25.210+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.041666666666666664
2023-09-01T13:38:25.211+05:30 [Info] Planner::finalizing the solution as final solution is found.
2023-09-01T13:38:25.211+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:25.211+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=1.  Elapsed Time=912us
2023-09-01T13:38:25.211+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=2.  Elapsed Time=8us
2023-09-01T13:38:25.211+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=3.  Elapsed Time=6us
2023-09-01T13:38:25.211+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=4.  Elapsed Time=5us
2023-09-01T13:38:25.211+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=5.  Elapsed Time=7us
2023-09-01T13:38:25.212+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=6.  Elapsed Time=6us
2023-09-01T13:38:25.212+05:30 [Warn] Unable to find a solution with resource constraint.  Relax resource constraint check.
2023-09-01T13:38:25.212+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.041666666666666664
2023-09-01T13:38:25.212+05:30 [Info] Planner::finalizing the solution as final solution is found.
2023-09-01T13:38:25.212+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Minimum memory test 9: single node rebalance with min memory > quota
2023-09-01T13:38:25.212+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=1.  Elapsed Time=5us
2023-09-01T13:38:25.212+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=2.  Elapsed Time=7us
2023-09-01T13:38:25.212+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=3.  Elapsed Time=6us
2023-09-01T13:38:25.212+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=4.  Elapsed Time=6us
2023-09-01T13:38:25.212+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=5.  Elapsed Time=5us
2023-09-01T13:38:25.212+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=6.  Elapsed Time=7us
2023-09-01T13:38:25.212+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=7.  Elapsed Time=6us
2023-09-01T13:38:25.212+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=8.  Elapsed Time=7us
2023-09-01T13:38:25.212+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=9.  Elapsed Time=6us
2023-09-01T13:38:25.213+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=10.  Elapsed Time=5us
2023-09-01T13:38:25.213+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=11.  Elapsed Time=7us
2023-09-01T13:38:25.213+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=12.  Elapsed Time=6us
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Minimum memory test 10: plan with partitioned index on empty cluster
2023-09-01T13:38:25.215+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 1.2247448713915892
2023-09-01T13:38:25.253+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 iterationTest :: Remove one node - failure
2023-09-01T13:38:25.254+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.39738460949458165
2023-09-01T13:38:25.345+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:25.345+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.019698007988524974
2023-09-01T13:38:25.345+05:30 [Info] Planner::initial temperature: initial resource variation 0.019698007988524974 temp 0.033030199201147505
2023-09-01T13:38:25.345+05:30 [Info] Planner::Running more iterations than 1 because of deleted nodes.
2023-09-01T13:38:25.346+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (2, 2) exceeded.
2023-09-01T13:38:25.346+05:30 [Info] Planner::finalizing the solution as the current resource variation is under control (0.21910630888170177).
2023-09-01T13:38:25.346+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=1.  Elapsed Time=515us
2023-09-01T13:38:25.346+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.019698007988524974
2023-09-01T13:38:25.346+05:30 [Info] Planner::initial temperature: initial resource variation 0.019698007988524974 temp 0.033030199201147505
2023-09-01T13:38:25.346+05:30 [Info] Planner::Running more iterations than 1 because of deleted nodes.
2023-09-01T13:38:25.346+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (2, 2) exceeded.
2023-09-01T13:38:25.346+05:30 [Info] Planner::finalizing the solution as the current resource variation is under control (0.16521812130018784).
2023-09-01T13:38:25.346+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=2.  Elapsed Time=440us
2023-09-01T13:38:25.346+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.019698007988524974
2023-09-01T13:38:25.346+05:30 [Info] Planner::initial temperature: initial resource variation 0.019698007988524974 temp 0.033030199201147505
2023-09-01T13:38:25.347+05:30 [Info] Planner::Running more iterations than 1 because of deleted nodes.
2023-09-01T13:38:25.347+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (2, 2) exceeded.
2023-09-01T13:38:25.347+05:30 [Info] Planner::finalizing the solution as the current resource variation is under control (0.14097573092916532).
2023-09-01T13:38:25.347+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=3.  Elapsed Time=430us
2023-09-01T13:38:25.347+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.019698007988524974
2023-09-01T13:38:25.347+05:30 [Info] Planner::initial temperature: initial resource variation 0.019698007988524974 temp 0.033030199201147505
2023-09-01T13:38:25.347+05:30 [Info] Planner::Running more iterations than 1 because of deleted nodes.
2023-09-01T13:38:25.347+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (2, 2) exceeded.
2023-09-01T13:38:25.347+05:30 [Info] Planner::finalizing the solution as the current resource variation is under control (0.16283676518147946).
2023-09-01T13:38:25.347+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=4.  Elapsed Time=453us
2023-09-01T13:38:25.347+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.019698007988524974
2023-09-01T13:38:25.347+05:30 [Info] Planner::initial temperature: initial resource variation 0.019698007988524974 temp 0.033030199201147505
2023-09-01T13:38:25.348+05:30 [Info] Planner::Running more iterations than 1 because of deleted nodes.
2023-09-01T13:38:25.348+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (2, 2) exceeded.
2023-09-01T13:38:25.348+05:30 [Info] Planner::finalizing the solution as the current resource variation is under control (0.16518866480308367).
2023-09-01T13:38:25.348+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=5.  Elapsed Time=447us
2023-09-01T13:38:25.348+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.019698007988524974
2023-09-01T13:38:25.348+05:30 [Info] Planner::initial temperature: initial resource variation 0.019698007988524974 temp 0.033030199201147505
2023-09-01T13:38:25.348+05:30 [Info] Planner::Running more iterations than 1 because of deleted nodes.
2023-09-01T13:38:25.348+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (2, 2) exceeded.
2023-09-01T13:38:25.348+05:30 [Info] Planner::finalizing the solution as the current resource variation is under control (0.22724976389151166).
2023-09-01T13:38:25.348+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=6.  Elapsed Time=442us
2023-09-01T13:38:25.348+05:30 [Warn] Unable to find a solution with resource constraint.  Relax resource constraint check.
2023-09-01T13:38:25.348+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.019698007988524974
2023-09-01T13:38:25.348+05:30 [Info] Planner::initial temperature: initial resource variation 0.019698007988524974 temp 0.033030199201147505
2023-09-01T13:38:25.349+05:30 [Info] Planner::Running more iterations than 1 because of deleted nodes.
2023-09-01T13:38:25.349+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (2, 2) exceeded.
2023-09-01T13:38:25.349+05:30 [Info] Planner::finalizing the solution as the current resource variation is under control (0.1926891757500282).
2023-09-01T13:38:25.349+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=7.  Elapsed Time=510us
2023-09-01T13:38:25.349+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.019698007988524974
2023-09-01T13:38:25.349+05:30 [Info] Planner::initial temperature: initial resource variation 0.019698007988524974 temp 0.033030199201147505
2023-09-01T13:38:25.349+05:30 [Info] Planner::Running more iterations than 1 because of deleted nodes.
2023-09-01T13:38:25.349+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (2, 2) exceeded.
2023-09-01T13:38:25.349+05:30 [Info] Planner::finalizing the solution as the current resource variation is under control (0.16518866480308367).
2023-09-01T13:38:25.349+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=8.  Elapsed Time=376us
2023-09-01T13:38:25.349+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.019698007988524974
2023-09-01T13:38:25.349+05:30 [Info] Planner::initial temperature: initial resource variation 0.019698007988524974 temp 0.033030199201147505
2023-09-01T13:38:25.350+05:30 [Info] Planner::Running more iterations than 1 because of deleted nodes.
2023-09-01T13:38:25.350+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (2, 2) exceeded.
2023-09-01T13:38:25.350+05:30 [Info] Planner::finalizing the solution as the current resource variation is under control (0.24018928910198656).
2023-09-01T13:38:25.350+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=9.  Elapsed Time=471us
2023-09-01T13:38:25.350+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.019698007988524974
2023-09-01T13:38:25.350+05:30 [Info] Planner::initial temperature: initial resource variation 0.019698007988524974 temp 0.033030199201147505
2023-09-01T13:38:25.350+05:30 [Info] Planner::Running more iterations than 1 because of deleted nodes.
2023-09-01T13:38:25.350+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (2, 2) exceeded.
2023-09-01T13:38:25.350+05:30 [Info] Planner::finalizing the solution as the current resource variation is under control (0.1997197451613064).
2023-09-01T13:38:25.350+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=10.  Elapsed Time=440us
2023-09-01T13:38:25.350+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.019698007988524974
2023-09-01T13:38:25.350+05:30 [Info] Planner::initial temperature: initial resource variation 0.019698007988524974 temp 0.033030199201147505
2023-09-01T13:38:25.351+05:30 [Info] Planner::Running more iterations than 1 because of deleted nodes.
2023-09-01T13:38:25.351+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (2, 2) exceeded.
2023-09-01T13:38:25.351+05:30 [Info] Planner::finalizing the solution as the current resource variation is under control (0.1997197451613064).
2023-09-01T13:38:25.351+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=11.  Elapsed Time=427us
2023-09-01T13:38:25.351+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.019698007988524974
2023-09-01T13:38:25.351+05:30 [Info] Planner::initial temperature: initial resource variation 0.019698007988524974 temp 0.033030199201147505
2023-09-01T13:38:25.351+05:30 [Info] Planner::Running more iterations than 1 because of deleted nodes.
2023-09-01T13:38:25.351+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (2, 2) exceeded.
2023-09-01T13:38:25.351+05:30 [Info] Planner::finalizing the solution as the current resource variation is under control (0.14097573092916532).
2023-09-01T13:38:25.351+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=12.  Elapsed Time=448us
2023-09-01T13:38:25.351+05:30 [Error] 
MemoryQuota: 5302940000000
CpuQuota: 6
--- Violations for index <5002464361137229003, bucket1, , > (mem 22.9467G, cpu 3.88232) at node 127.0.0.1:9001 
	Can move to 127.0.0.1:9002: NoViolation (free mem 4.64914T, free cpu -36.19207999999999)
	Can move to 127.0.0.1:9003: NoViolation (free mem 4.62806T, free cpu -35.886120000000005)
--- Violations for index <16258497398424723417, bucket1, , > (mem 21.3884G, cpu 2.99124) at node 127.0.0.1:9001 
	Can move to 127.0.0.1:9002: NoViolation (free mem 4.64914T, free cpu -36.19207999999999)
	Can move to 127.0.0.1:9003: NoViolation (free mem 4.62806T, free cpu -35.886120000000005)
--- Violations for index <6534695437485591241, bucket1, , > (mem 21.4744G, cpu 3.95676) at node 127.0.0.1:9001 
	Can move to 127.0.0.1:9002: NoViolation (free mem 4.64914T, free cpu -36.19207999999999)
	Can move to 127.0.0.1:9003: NoViolation (free mem 4.62806T, free cpu -35.886120000000005)
--- Violations for index <11478798686444503340, bucket1, , > (mem 17.7335G, cpu 2.64844) at node 127.0.0.1:9001 
	Can move to 127.0.0.1:9002: NoViolation (free mem 4.64914T, free cpu -36.19207999999999)
	Can move to 127.0.0.1:9003: NoViolation (free mem 4.62806T, free cpu -35.886120000000005)
--- Violations for index <2927755889872266961, bucket1, , > (mem 19.1015G, cpu 2.30484) at node 127.0.0.1:9001 
	Can move to 127.0.0.1:9002: NoViolation (free mem 4.64914T, free cpu -36.19207999999999)
	Can move to 127.0.0.1:9003: NoViolation (free mem 4.62806T, free cpu -35.886120000000005)
--- Violations for index <2083057437900411658, bucket1, , > (mem 20.3552G, cpu 4.8430800000000005) at node 127.0.0.1:9001 
	Can move to 127.0.0.1:9002: NoViolation (free mem 4.64914T, free cpu -36.19207999999999)
	Can move to 127.0.0.1:9003: NoViolation (free mem 4.62806T, free cpu -35.886120000000005)
--- Violations for index <5625199838968122080, bucket1, , > (mem 14.3197G, cpu 4.79304) at node 127.0.0.1:9001 
	Can move to 127.0.0.1:9002: NoViolation (free mem 4.64914T, free cpu -36.19207999999999)
	Can move to 127.0.0.1:9003: NoViolation (free mem 4.62806T, free cpu -35.886120000000005)
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 iterationTest :: Remove one node - success
2023-09-01T13:38:25.352+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.18511841007258734
2023-09-01T13:38:25.431+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:25.431+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.008398869595003598
2023-09-01T13:38:25.431+05:30 [Info] Planner::initial temperature: initial resource variation 0.008398869595003598 temp 0.024160113040499642
2023-09-01T13:38:25.431+05:30 [Info] Planner::Running more iterations than 1 because of deleted nodes.
2023-09-01T13:38:25.431+05:30 [Info] Planner::Finished planner run after 10 iterations.
2023-09-01T13:38:25.431+05:30 [Info] Planner::finalizing the solution as the current resource variation is under control (0.031097355672212675).
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 iterationTest :: Index rebuild - success
2023-09-01T13:38:25.434+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i13,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.434+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i15,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.434+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i18,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.434+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i22,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.434+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i0,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.434+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i5,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.434+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i10,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.434+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i24,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.434+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i26,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.434+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i28,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.434+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i2,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.435+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i4,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.435+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i8,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.435+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i11,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.435+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i27,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.435+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i1,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.435+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i3,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.435+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i6,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.435+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i12,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.435+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i14,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.435+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i20,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.435+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i23,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.435+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i19,1,0) with initial placement on node: 172.16.30.75:9000
2023-09-01T13:38:25.435+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i21,1,0) with initial placement on node: 127.0.0.1:9001
2023-09-01T13:38:25.435+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i25,1,0) with initial placement on node: 127.0.0.1:9001
2023-09-01T13:38:25.435+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i29,1,0) with initial placement on node: 172.16.30.75:9000
2023-09-01T13:38:25.435+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i7,1,0) with initial placement on node: 172.16.30.75:9000
2023-09-01T13:38:25.435+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i9,1,0) with initial placement on node: 172.16.30.75:9000
2023-09-01T13:38:25.435+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i16,1,0) with initial placement on node: 172.16.30.75:9000
2023-09-01T13:38:25.435+05:30 [Info] Rebuilding lost replica for (default,_default,_default,i17,1,0) with initial placement on node: 172.16.30.75:9000
2023-09-01T13:38:25.436+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.5944184833375669
2023-09-01T13:38:25.436+05:30 [Info] Planner::planSingleRun Skip running planner as current solution resource variation: 0.5944184833375669 is less than threshold: 0.8. No nodes have been added or deleted and there are no violations observed
2023-09-01T13:38:25.436+05:30 [Info] Planner::initial temperature: initial resource variation 0.5944184833375669 temp 0.02055815166624332
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 iterationTest :: Index rebuild - initial placement - success
2023-09-01T13:38:25.437+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 127.0.0.1:9000
2023-09-01T13:38:25.437+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 127.0.0.1:9001
2023-09-01T13:38:25.437+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 127.0.0.1:9002
2023-09-01T13:38:25.437+05:30 [Info] Rebuilding lost replica for (default,,,idx1,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.437+05:30 [Info] Rebuilding lost replica for (default,,,idx2,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.437+05:30 [Info] Rebuilding lost replica for (default,,,idx3,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.437+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.6236095644623236
2023-09-01T13:38:25.437+05:30 [Info] Planner::initial temperature: initial resource variation 0.6236095644623236 temp 0.03663904355376764
2023-09-01T13:38:25.437+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (0, 0) exceeded.
2023-09-01T13:38:25.437+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:25.437+05:30 [Info] Planner::finalizing the solution as the current resource variation is under control (0.6236095644623236).
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 iterationTest :: Index rebuild - initial placement - numReplica > numSG - success
2023-09-01T13:38:25.438+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 127.0.0.1:9000
2023-09-01T13:38:25.438+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 127.0.0.1:9001
2023-09-01T13:38:25.438+05:30 [Info] Planner::setExcludeInForNonEmptyNodes Set Exclude=in for 127.0.0.1:9002
2023-09-01T13:38:25.438+05:30 [Info] Rebuilding lost replica for (default,,,idx1,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.438+05:30 [Info] Rebuilding lost replica for (default,,,idx1,2,0) with initial placement on node: 127.0.0.1:9001
2023-09-01T13:38:25.438+05:30 [Info] Rebuilding lost replica for (default,,,idx2,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.438+05:30 [Info] Rebuilding lost replica for (default,,,idx2,2,0) with initial placement on node: 127.0.0.1:9000
2023-09-01T13:38:25.438+05:30 [Info] Rebuilding lost replica for (default,,,idx3,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:25.438+05:30 [Info] Rebuilding lost replica for (default,,,idx3,2,0) with initial placement on node: 127.0.0.1:9000
2023-09-01T13:38:25.438+05:30 [Warn] Index has more replica than server group. Index=idx3 0 (replica 2) Bucket=default Scope= Collection=
2023-09-01T13:38:25.438+05:30 [Warn] Index has more replica than server group. Index=idx1 0 (replica 2) Bucket=default Scope= Collection=
2023-09-01T13:38:25.438+05:30 [Warn] Index has more replica than server group. Index=idx2 0 (replica 2) Bucket=default Scope= Collection=
2023-09-01T13:38:25.438+05:30 [Warn] Index has more replica than server group. Index=idx3 0 Bucket=default Scope= Collection=
2023-09-01T13:38:25.438+05:30 [Warn] Index has more replica than server group. Index=idx1 0 (replica 1) Bucket=default Scope= Collection=
2023-09-01T13:38:25.438+05:30 [Warn] Index has more replica than server group. Index=idx2 0 (replica 1) Bucket=default Scope= Collection=
2023-09-01T13:38:25.438+05:30 [Warn] Index has more replica than server group. Index=idx3 0 (replica 1) Bucket=default Scope= Collection=
2023-09-01T13:38:25.438+05:30 [Warn] Index has more replica than server group. Index=idx1 0 Bucket=default Scope= Collection=
2023-09-01T13:38:25.438+05:30 [Warn] Index has more replica than server group. Index=idx2 0 Bucket=default Scope= Collection=
2023-09-01T13:38:25.439+05:30 [Warn] Index has more replica than server group. Index=idx3 0 (replica 1) Bucket=default Scope= Collection=
2023-09-01T13:38:25.439+05:30 [Warn] Index has more replica than server group. Index=idx1 0 Bucket=default Scope= Collection=
2023-09-01T13:38:25.439+05:30 [Warn] Index has more replica than server group. Index=idx2 0 Bucket=default Scope= Collection=
2023-09-01T13:38:25.439+05:30 [Warn] Index has more replica than server group. Index=idx3 0 Bucket=default Scope= Collection=
2023-09-01T13:38:25.439+05:30 [Warn] Index has more replica than server group. Index=idx1 0 (replica 1) Bucket=default Scope= Collection=
2023-09-01T13:38:25.439+05:30 [Warn] Index has more replica than server group. Index=idx2 0 (replica 1) Bucket=default Scope= Collection=
2023-09-01T13:38:25.439+05:30 [Warn] Index has more replica than server group. Index=idx1 0 (replica 2) Bucket=default Scope= Collection=
2023-09-01T13:38:25.439+05:30 [Warn] Index has more replica than server group. Index=idx2 0 (replica 2) Bucket=default Scope= Collection=
2023-09-01T13:38:25.439+05:30 [Warn] Index has more replica than server group. Index=idx3 0 (replica 2) Bucket=default Scope= Collection=
2023-09-01T13:38:25.439+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.3333333333333333
2023-09-01T13:38:25.439+05:30 [Info] Planner::initial temperature: initial resource variation 0.3333333333333333 temp 0.06566666666666668
2023-09-01T13:38:25.439+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (0, 0) exceeded.
2023-09-01T13:38:25.439+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:25.439+05:30 [Info] Planner::finalizing the solution as the current resource variation is under control (0.3333333333333333).
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 scaling with excludeIn - 8 identical index, add 1, 1x
2023-09-01T13:38:25.439+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.5
2023-09-01T13:38:25.439+05:30 [Info] Planner::initial temperature: initial resource variation 0.5 temp 0.05
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 swap with excludeIn - 8 identical index, add 1, 1x
2023-09-01T13:38:25.946+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.5
2023-09-01T13:38:25.946+05:30 [Info] Planner::initial temperature: initial resource variation 0.5 temp 0.05
2023-09-01T13:38:25.946+05:30 [Info] Planner::finalizing the solution as final solution is found.
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 replica repair with excludeIn - 3 replica, 4 zone, delete 1, 1x
2023-09-01T13:38:25.947+05:30 [Info] Planner::planSingleRun Initial variance of the solution: NaN
2023-09-01T13:38:25.947+05:30 [Info] Planner::initial temperature: initial resource variation NaN temp NaN
2023-09-01T13:38:25.947+05:30 [Info] Planner::finalizing the solution as final solution is found.
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 heterogenous rebalance - keep vertically scaled Node, swap 1, 1x
2023-09-01T13:38:25.948+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 1.1237823254333867
2023-09-01T13:38:25.948+05:30 [Info] Planner::initial temperature: initial resource variation 1.1237823254333867 temp 0.11237823254333867
2023-09-01T13:38:25.949+05:30 [Info] Planner::finalizing the solution as final solution is found.
--- PASS: TestPlanner (12.40s)
=== RUN   TestGreedyPlanner
2023/09/01 13:38:25 In TestGreedyPlanner()
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Place Single Index Instance - 3 empty nodes - 1 SG
2023-09-01T13:38:25.953+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Place Single Index Instance - 2 empty nodes, 1 non-empty node - 1 SG
2023-09-01T13:38:25.955+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Place Single Index Instance - 1 empty node, 2 non-empty nodes - 1 SG
2023-09-01T13:38:25.957+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Place Single Index Instance - 3 non-empty nodes - 1 SG
2023-09-01T13:38:25.960+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Place Index With 1 Replica - 3 empty nodes - 1 SG
2023-09-01T13:38:25.962+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Place Index With 1 Replica - 2 empty nodes, 1 non-empty node - 1 SG
2023-09-01T13:38:25.964+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Place Index With 1 Replica - 1 empty node, 2 non-empty nodes - 1 SG
2023-09-01T13:38:25.966+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Place Index With 1 Replica - 3 non-empty nodes - 1 SG
2023-09-01T13:38:25.967+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Place Index With 2 Replica - 3 empty nodes - 1 SG
2023-09-01T13:38:25.969+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Place Index With 2 Replica - 3 non-empty nodes - 1 SG
2023-09-01T13:38:25.971+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Place Index With 1 Replica - 2 empty nodes, 1 non-empty node - 2 SG
2023-09-01T13:38:25.973+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Place Index With 1 Replica - 1 empty node, 2 non-empty nodes - 2 SG
2023-09-01T13:38:25.975+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Place Index With 1 Replica - 3 non-empty nodes - 2 SG
2023-09-01T13:38:25.977+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Place Equivalent Index Without any replica - 3 non-empty nodes - 1 SG
2023-09-01T13:38:25.979+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Place Equivalent Index With 1 Replica - 3 non-empty nodes - 1 SG - Skip least loaded node
2023-09-01T13:38:25.981+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Place Equivalent Index With 1 Replica - 3 non-empty nodes - 1 SG - Use least loaded node
2023-09-01T13:38:25.983+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:25 -------------------------------------------
2023/09/01 13:38:25 Place 60 index instaces on 3 empty nodes - 1 SG
2023-09-01T13:38:25.986+05:30 [Info] Using greedy index placement for index 1000987654
2023-09-01T13:38:25.987+05:30 [Info] Using greedy index placement for index 1001987654
2023-09-01T13:38:25.989+05:30 [Info] Using greedy index placement for index 1002987654
2023-09-01T13:38:25.991+05:30 [Info] Using greedy index placement for index 1003987654
2023-09-01T13:38:25.992+05:30 [Info] Using greedy index placement for index 1004987654
2023-09-01T13:38:25.994+05:30 [Info] Using greedy index placement for index 1005987654
2023-09-01T13:38:25.995+05:30 [Info] Using greedy index placement for index 1006987654
2023-09-01T13:38:25.997+05:30 [Info] Using greedy index placement for index 1007987654
2023-09-01T13:38:25.999+05:30 [Info] Using greedy index placement for index 1008987654
2023-09-01T13:38:26.000+05:30 [Info] Using greedy index placement for index 1009987654
2023-09-01T13:38:26.002+05:30 [Info] Using greedy index placement for index 1010987654
2023-09-01T13:38:26.004+05:30 [Info] Using greedy index placement for index 1011987654
2023-09-01T13:38:26.005+05:30 [Info] Using greedy index placement for index 1012987654
2023-09-01T13:38:26.007+05:30 [Info] Using greedy index placement for index 1013987654
2023-09-01T13:38:26.009+05:30 [Info] Using greedy index placement for index 1014987654
2023-09-01T13:38:26.010+05:30 [Info] Using greedy index placement for index 1015987654
2023-09-01T13:38:26.012+05:30 [Info] Using greedy index placement for index 1016987654
2023-09-01T13:38:26.014+05:30 [Info] Using greedy index placement for index 1017987654
2023-09-01T13:38:26.015+05:30 [Info] Using greedy index placement for index 1018987654
2023-09-01T13:38:26.017+05:30 [Info] Using greedy index placement for index 1019987654
2023-09-01T13:38:26.019+05:30 [Info] Using greedy index placement for index 1020987654
2023-09-01T13:38:26.020+05:30 [Info] Using greedy index placement for index 1021987654
2023-09-01T13:38:26.022+05:30 [Info] Using greedy index placement for index 1022987654
2023-09-01T13:38:26.024+05:30 [Info] Using greedy index placement for index 1023987654
2023-09-01T13:38:26.025+05:30 [Info] Using greedy index placement for index 1024987654
2023-09-01T13:38:26.027+05:30 [Info] Using greedy index placement for index 1025987654
2023-09-01T13:38:26.029+05:30 [Info] Using greedy index placement for index 1026987654
2023-09-01T13:38:26.030+05:30 [Info] Using greedy index placement for index 1027987654
2023-09-01T13:38:26.032+05:30 [Info] Using greedy index placement for index 1028987654
2023-09-01T13:38:26.034+05:30 [Info] Using greedy index placement for index 1029987654
2023-09-01T13:38:26.036+05:30 [Info] Using greedy index placement for index 1030987654
2023-09-01T13:38:26.037+05:30 [Info] Using greedy index placement for index 1031987654
2023-09-01T13:38:26.039+05:30 [Info] Using greedy index placement for index 1032987654
2023-09-01T13:38:26.041+05:30 [Info] Using greedy index placement for index 1033987654
2023-09-01T13:38:26.042+05:30 [Info] Using greedy index placement for index 1034987654
2023-09-01T13:38:26.044+05:30 [Info] Using greedy index placement for index 1035987654
2023-09-01T13:38:26.046+05:30 [Info] Using greedy index placement for index 1036987654
2023-09-01T13:38:26.048+05:30 [Info] Using greedy index placement for index 1037987654
2023-09-01T13:38:26.050+05:30 [Info] Using greedy index placement for index 1038987654
2023-09-01T13:38:26.051+05:30 [Info] Using greedy index placement for index 1039987654
2023-09-01T13:38:26.053+05:30 [Info] Using greedy index placement for index 1040987654
2023-09-01T13:38:26.055+05:30 [Info] Using greedy index placement for index 1041987654
2023-09-01T13:38:26.057+05:30 [Info] Using greedy index placement for index 1042987654
2023-09-01T13:38:26.058+05:30 [Info] Using greedy index placement for index 1043987654
2023-09-01T13:38:26.060+05:30 [Info] Using greedy index placement for index 1044987654
2023-09-01T13:38:26.062+05:30 [Info] Using greedy index placement for index 1045987654
2023-09-01T13:38:26.063+05:30 [Info] Using greedy index placement for index 1046987654
2023-09-01T13:38:26.065+05:30 [Info] Using greedy index placement for index 1047987654
2023-09-01T13:38:26.067+05:30 [Info] Using greedy index placement for index 1048987654
2023-09-01T13:38:26.069+05:30 [Info] Using greedy index placement for index 1049987654
2023-09-01T13:38:26.070+05:30 [Info] Using greedy index placement for index 1050987654
2023-09-01T13:38:26.072+05:30 [Info] Using greedy index placement for index 1051987654
2023-09-01T13:38:26.074+05:30 [Info] Using greedy index placement for index 1052987654
2023-09-01T13:38:26.076+05:30 [Info] Using greedy index placement for index 1053987654
2023-09-01T13:38:26.077+05:30 [Info] Using greedy index placement for index 1054987654
2023-09-01T13:38:26.079+05:30 [Info] Using greedy index placement for index 1055987654
2023-09-01T13:38:26.081+05:30 [Info] Using greedy index placement for index 1056987654
2023-09-01T13:38:26.083+05:30 [Info] Using greedy index placement for index 1057987654
2023-09-01T13:38:26.084+05:30 [Info] Using greedy index placement for index 1058987654
2023-09-01T13:38:26.086+05:30 [Info] Using greedy index placement for index 1059987654
2023-09-01T13:38:26.086+05:30 [Info] Actual variance of deferred index count across nodes is 0
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place 60 index instaces on 1 empty and 1 10 percent filled node - 1 SG
2023-09-01T13:38:26.089+05:30 [Info] Using greedy index placement for index 1000987654
2023-09-01T13:38:26.090+05:30 [Info] Using greedy index placement for index 1001987654
2023-09-01T13:38:26.092+05:30 [Info] Using greedy index placement for index 1002987654
2023-09-01T13:38:26.094+05:30 [Info] Using greedy index placement for index 1003987654
2023-09-01T13:38:26.095+05:30 [Info] Using greedy index placement for index 1004987654
2023-09-01T13:38:26.097+05:30 [Info] Using greedy index placement for index 1005987654
2023-09-01T13:38:26.099+05:30 [Info] Using greedy index placement for index 1006987654
2023-09-01T13:38:26.100+05:30 [Info] Using greedy index placement for index 1007987654
2023-09-01T13:38:26.102+05:30 [Info] Using greedy index placement for index 1008987654
2023-09-01T13:38:26.104+05:30 [Info] Using greedy index placement for index 1009987654
2023-09-01T13:38:26.106+05:30 [Info] Using greedy index placement for index 1010987654
2023-09-01T13:38:26.107+05:30 [Info] Using greedy index placement for index 1011987654
2023-09-01T13:38:26.109+05:30 [Info] Using greedy index placement for index 1012987654
2023-09-01T13:38:26.112+05:30 [Info] Using greedy index placement for index 1013987654
2023-09-01T13:38:26.114+05:30 [Info] Using greedy index placement for index 1014987654
2023-09-01T13:38:26.116+05:30 [Info] Using greedy index placement for index 1015987654
2023-09-01T13:38:26.118+05:30 [Info] Using greedy index placement for index 1016987654
2023-09-01T13:38:26.120+05:30 [Info] Using greedy index placement for index 1017987654
2023-09-01T13:38:26.122+05:30 [Info] Using greedy index placement for index 1018987654
2023-09-01T13:38:26.124+05:30 [Info] Using greedy index placement for index 1019987654
2023-09-01T13:38:26.126+05:30 [Info] Using greedy index placement for index 1020987654
2023-09-01T13:38:26.128+05:30 [Info] Using greedy index placement for index 1021987654
2023-09-01T13:38:26.130+05:30 [Info] Using greedy index placement for index 1022987654
2023-09-01T13:38:26.132+05:30 [Info] Using greedy index placement for index 1023987654
2023-09-01T13:38:26.133+05:30 [Info] Using greedy index placement for index 1024987654
2023-09-01T13:38:26.135+05:30 [Info] Using greedy index placement for index 1025987654
2023-09-01T13:38:26.137+05:30 [Info] Using greedy index placement for index 1026987654
2023-09-01T13:38:26.139+05:30 [Info] Using greedy index placement for index 1027987654
2023-09-01T13:38:26.141+05:30 [Info] Using greedy index placement for index 1028987654
2023-09-01T13:38:26.143+05:30 [Info] Using greedy index placement for index 1029987654
2023-09-01T13:38:26.145+05:30 [Info] Using greedy index placement for index 1030987654
2023-09-01T13:38:26.147+05:30 [Info] Using greedy index placement for index 1031987654
2023-09-01T13:38:26.149+05:30 [Info] Using greedy index placement for index 1032987654
2023-09-01T13:38:26.151+05:30 [Info] Using greedy index placement for index 1033987654
2023-09-01T13:38:26.153+05:30 [Info] Using greedy index placement for index 1034987654
2023-09-01T13:38:26.155+05:30 [Info] Using greedy index placement for index 1035987654
2023-09-01T13:38:26.157+05:30 [Info] Using greedy index placement for index 1036987654
2023-09-01T13:38:26.158+05:30 [Info] Using greedy index placement for index 1037987654
2023-09-01T13:38:26.161+05:30 [Info] Using greedy index placement for index 1038987654
2023-09-01T13:38:26.168+05:30 [Info] Using greedy index placement for index 1039987654
2023-09-01T13:38:26.170+05:30 [Info] Using greedy index placement for index 1040987654
2023-09-01T13:38:26.171+05:30 [Info] Using greedy index placement for index 1041987654
2023-09-01T13:38:26.174+05:30 [Info] Using greedy index placement for index 1042987654
2023-09-01T13:38:26.175+05:30 [Info] Using greedy index placement for index 1043987654
2023-09-01T13:38:26.177+05:30 [Info] Using greedy index placement for index 1044987654
2023-09-01T13:38:26.179+05:30 [Info] Using greedy index placement for index 1045987654
2023-09-01T13:38:26.181+05:30 [Info] Using greedy index placement for index 1046987654
2023-09-01T13:38:26.182+05:30 [Info] Using greedy index placement for index 1047987654
2023-09-01T13:38:26.184+05:30 [Info] Using greedy index placement for index 1048987654
2023-09-01T13:38:26.186+05:30 [Info] Using greedy index placement for index 1049987654
2023-09-01T13:38:26.188+05:30 [Info] Using greedy index placement for index 1050987654
2023-09-01T13:38:26.190+05:30 [Info] Using greedy index placement for index 1051987654
2023-09-01T13:38:26.192+05:30 [Info] Using greedy index placement for index 1052987654
2023-09-01T13:38:26.193+05:30 [Info] Using greedy index placement for index 1053987654
2023-09-01T13:38:26.195+05:30 [Info] Using greedy index placement for index 1054987654
2023-09-01T13:38:26.197+05:30 [Info] Using greedy index placement for index 1055987654
2023-09-01T13:38:26.199+05:30 [Info] Using greedy index placement for index 1056987654
2023-09-01T13:38:26.201+05:30 [Info] Using greedy index placement for index 1057987654
2023-09-01T13:38:26.203+05:30 [Info] Using greedy index placement for index 1058987654
2023-09-01T13:38:26.204+05:30 [Info] Using greedy index placement for index 1059987654
2023-09-01T13:38:26.204+05:30 [Info] Actual variance of deferred index count across nodes is 8
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place 60 index instaces on 1 empty and 1 30 percent filled node - 1 SG
2023-09-01T13:38:26.207+05:30 [Info] Using greedy index placement for index 1000987654
2023-09-01T13:38:26.208+05:30 [Info] Using greedy index placement for index 1001987654
2023-09-01T13:38:26.210+05:30 [Info] Using greedy index placement for index 1002987654
2023-09-01T13:38:26.212+05:30 [Info] Using greedy index placement for index 1003987654
2023-09-01T13:38:26.213+05:30 [Info] Using greedy index placement for index 1004987654
2023-09-01T13:38:26.215+05:30 [Info] Using greedy index placement for index 1005987654
2023-09-01T13:38:26.217+05:30 [Info] Using greedy index placement for index 1006987654
2023-09-01T13:38:26.218+05:30 [Info] Using greedy index placement for index 1007987654
2023-09-01T13:38:26.220+05:30 [Info] Using greedy index placement for index 1008987654
2023-09-01T13:38:26.222+05:30 [Info] Using greedy index placement for index 1009987654
2023-09-01T13:38:26.223+05:30 [Info] Using greedy index placement for index 1010987654
2023-09-01T13:38:26.225+05:30 [Info] Using greedy index placement for index 1011987654
2023-09-01T13:38:26.227+05:30 [Info] Using greedy index placement for index 1012987654
2023-09-01T13:38:26.229+05:30 [Info] Using greedy index placement for index 1013987654
2023-09-01T13:38:26.230+05:30 [Info] Using greedy index placement for index 1014987654
2023-09-01T13:38:26.232+05:30 [Info] Using greedy index placement for index 1015987654
2023-09-01T13:38:26.234+05:30 [Info] Using greedy index placement for index 1016987654
2023-09-01T13:38:26.236+05:30 [Info] Using greedy index placement for index 1017987654
2023-09-01T13:38:26.238+05:30 [Info] Using greedy index placement for index 1018987654
2023-09-01T13:38:26.239+05:30 [Info] Using greedy index placement for index 1019987654
2023-09-01T13:38:26.241+05:30 [Info] Using greedy index placement for index 1020987654
2023-09-01T13:38:26.243+05:30 [Info] Using greedy index placement for index 1021987654
2023-09-01T13:38:26.246+05:30 [Info] Using greedy index placement for index 1022987654
2023-09-01T13:38:26.247+05:30 [Info] Using greedy index placement for index 1023987654
2023-09-01T13:38:26.249+05:30 [Info] Using greedy index placement for index 1024987654
2023-09-01T13:38:26.250+05:30 [Info] Using greedy index placement for index 1025987654
2023-09-01T13:38:26.252+05:30 [Info] Using greedy index placement for index 1026987654
2023-09-01T13:38:26.254+05:30 [Info] Using greedy index placement for index 1027987654
2023-09-01T13:38:26.255+05:30 [Info] Using greedy index placement for index 1028987654
2023-09-01T13:38:26.257+05:30 [Info] Using greedy index placement for index 1029987654
2023-09-01T13:38:26.259+05:30 [Info] Using greedy index placement for index 1030987654
2023-09-01T13:38:26.260+05:30 [Info] Using greedy index placement for index 1031987654
2023-09-01T13:38:26.262+05:30 [Info] Using greedy index placement for index 1032987654
2023-09-01T13:38:26.264+05:30 [Info] Using greedy index placement for index 1033987654
2023-09-01T13:38:26.266+05:30 [Info] Using greedy index placement for index 1034987654
2023-09-01T13:38:26.267+05:30 [Info] Using greedy index placement for index 1035987654
2023-09-01T13:38:26.269+05:30 [Info] Using greedy index placement for index 1036987654
2023-09-01T13:38:26.271+05:30 [Info] Using greedy index placement for index 1037987654
2023-09-01T13:38:26.273+05:30 [Info] Using greedy index placement for index 1038987654
2023-09-01T13:38:26.274+05:30 [Info] Using greedy index placement for index 1039987654
2023-09-01T13:38:26.276+05:30 [Info] Using greedy index placement for index 1040987654
2023-09-01T13:38:26.278+05:30 [Info] Using greedy index placement for index 1041987654
2023-09-01T13:38:26.280+05:30 [Info] Using greedy index placement for index 1042987654
2023-09-01T13:38:26.281+05:30 [Info] Using greedy index placement for index 1043987654
2023-09-01T13:38:26.283+05:30 [Info] Using greedy index placement for index 1044987654
2023-09-01T13:38:26.285+05:30 [Info] Using greedy index placement for index 1045987654
2023-09-01T13:38:26.287+05:30 [Info] Using greedy index placement for index 1046987654
2023-09-01T13:38:26.289+05:30 [Info] Using greedy index placement for index 1047987654
2023-09-01T13:38:26.290+05:30 [Info] Using greedy index placement for index 1048987654
2023-09-01T13:38:26.292+05:30 [Info] Using greedy index placement for index 1049987654
2023-09-01T13:38:26.294+05:30 [Info] Using greedy index placement for index 1050987654
2023-09-01T13:38:26.296+05:30 [Info] Using greedy index placement for index 1051987654
2023-09-01T13:38:26.298+05:30 [Info] Using greedy index placement for index 1052987654
2023-09-01T13:38:26.299+05:30 [Info] Using greedy index placement for index 1053987654
2023-09-01T13:38:26.301+05:30 [Info] Using greedy index placement for index 1054987654
2023-09-01T13:38:26.303+05:30 [Info] Using greedy index placement for index 1055987654
2023-09-01T13:38:26.305+05:30 [Info] Using greedy index placement for index 1056987654
2023-09-01T13:38:26.307+05:30 [Info] Using greedy index placement for index 1057987654
2023-09-01T13:38:26.309+05:30 [Info] Using greedy index placement for index 1058987654
2023-09-01T13:38:26.311+05:30 [Info] Using greedy index placement for index 1059987654
2023-09-01T13:38:26.311+05:30 [Info] Actual variance of deferred index count across nodes is 98
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place 5 index instaces on 1 empty and 1 60 percent filled node - 1 SG
2023-09-01T13:38:26.314+05:30 [Info] Using greedy index placement for index 1000987654
2023-09-01T13:38:26.316+05:30 [Info] Using greedy index placement for index 1001987654
2023-09-01T13:38:26.317+05:30 [Info] Using greedy index placement for index 1002987654
2023-09-01T13:38:26.319+05:30 [Info] Using greedy index placement for index 1003987654
2023-09-01T13:38:26.321+05:30 [Info] Using greedy index placement for index 1004987654
2023-09-01T13:38:26.321+05:30 [Info] Actual variance of deferred index count across nodes is 4.5
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place 60 index instaces on 1 empty and 1 60 percent filled node - 1 SG
2023-09-01T13:38:26.322+05:30 [Info] Using greedy index placement for index 1000987654
2023-09-01T13:38:26.324+05:30 [Info] Using greedy index placement for index 1001987654
2023-09-01T13:38:26.326+05:30 [Info] Using greedy index placement for index 1002987654
2023-09-01T13:38:26.328+05:30 [Info] Using greedy index placement for index 1003987654
2023-09-01T13:38:26.329+05:30 [Info] Using greedy index placement for index 1004987654
2023-09-01T13:38:26.331+05:30 [Info] Using greedy index placement for index 1005987654
2023-09-01T13:38:26.332+05:30 [Info] Using greedy index placement for index 1006987654
2023-09-01T13:38:26.334+05:30 [Info] Using greedy index placement for index 1007987654
2023-09-01T13:38:26.336+05:30 [Info] Using greedy index placement for index 1008987654
2023-09-01T13:38:26.338+05:30 [Info] Using greedy index placement for index 1009987654
2023-09-01T13:38:26.339+05:30 [Info] Using greedy index placement for index 1010987654
2023-09-01T13:38:26.341+05:30 [Info] Using greedy index placement for index 1011987654
2023-09-01T13:38:26.342+05:30 [Info] Using greedy index placement for index 1012987654
2023-09-01T13:38:26.344+05:30 [Info] Using greedy index placement for index 1013987654
2023-09-01T13:38:26.346+05:30 [Info] Using greedy index placement for index 1014987654
2023-09-01T13:38:26.347+05:30 [Info] Using greedy index placement for index 1015987654
2023-09-01T13:38:26.349+05:30 [Info] Using greedy index placement for index 1016987654
2023-09-01T13:38:26.351+05:30 [Info] Using greedy index placement for index 1017987654
2023-09-01T13:38:26.352+05:30 [Info] Using greedy index placement for index 1018987654
2023-09-01T13:38:26.354+05:30 [Info] Using greedy index placement for index 1019987654
2023-09-01T13:38:26.355+05:30 [Info] Using greedy index placement for index 1020987654
2023-09-01T13:38:26.357+05:30 [Info] Using greedy index placement for index 1021987654
2023-09-01T13:38:26.359+05:30 [Info] Using greedy index placement for index 1022987654
2023-09-01T13:38:26.360+05:30 [Info] Using greedy index placement for index 1023987654
2023-09-01T13:38:26.362+05:30 [Info] Using greedy index placement for index 1024987654
2023-09-01T13:38:26.364+05:30 [Info] Using greedy index placement for index 1025987654
2023-09-01T13:38:26.366+05:30 [Info] Using greedy index placement for index 1026987654
2023-09-01T13:38:26.367+05:30 [Info] Using greedy index placement for index 1027987654
2023-09-01T13:38:26.369+05:30 [Info] Using greedy index placement for index 1028987654
2023-09-01T13:38:26.371+05:30 [Info] Using greedy index placement for index 1029987654
2023-09-01T13:38:26.372+05:30 [Info] Using greedy index placement for index 1030987654
2023-09-01T13:38:26.374+05:30 [Info] Using greedy index placement for index 1031987654
2023-09-01T13:38:26.376+05:30 [Info] Using greedy index placement for index 1032987654
2023-09-01T13:38:26.377+05:30 [Info] Using greedy index placement for index 1033987654
2023-09-01T13:38:26.379+05:30 [Info] Using greedy index placement for index 1034987654
2023-09-01T13:38:26.381+05:30 [Info] Using greedy index placement for index 1035987654
2023-09-01T13:38:26.383+05:30 [Info] Using greedy index placement for index 1036987654
2023-09-01T13:38:26.384+05:30 [Info] Using greedy index placement for index 1037987654
2023-09-01T13:38:26.386+05:30 [Info] Using greedy index placement for index 1038987654
2023-09-01T13:38:26.388+05:30 [Info] Using greedy index placement for index 1039987654
2023-09-01T13:38:26.390+05:30 [Info] Using greedy index placement for index 1040987654
2023-09-01T13:38:26.392+05:30 [Info] Using greedy index placement for index 1041987654
2023-09-01T13:38:26.393+05:30 [Info] Using greedy index placement for index 1042987654
2023-09-01T13:38:26.395+05:30 [Info] Using greedy index placement for index 1043987654
2023-09-01T13:38:26.397+05:30 [Info] Using greedy index placement for index 1044987654
2023-09-01T13:38:26.398+05:30 [Info] Using greedy index placement for index 1045987654
2023-09-01T13:38:26.400+05:30 [Info] Using greedy index placement for index 1046987654
2023-09-01T13:38:26.402+05:30 [Info] Using greedy index placement for index 1047987654
2023-09-01T13:38:26.403+05:30 [Info] Using greedy index placement for index 1048987654
2023-09-01T13:38:26.405+05:30 [Info] Using greedy index placement for index 1049987654
2023-09-01T13:38:26.407+05:30 [Info] Using greedy index placement for index 1050987654
2023-09-01T13:38:26.409+05:30 [Info] Using greedy index placement for index 1051987654
2023-09-01T13:38:26.410+05:30 [Info] Using greedy index placement for index 1052987654
2023-09-01T13:38:26.412+05:30 [Info] Using greedy index placement for index 1053987654
2023-09-01T13:38:26.414+05:30 [Info] Using greedy index placement for index 1054987654
2023-09-01T13:38:26.416+05:30 [Info] Using greedy index placement for index 1055987654
2023-09-01T13:38:26.418+05:30 [Info] Using greedy index placement for index 1056987654
2023-09-01T13:38:26.420+05:30 [Info] Using greedy index placement for index 1057987654
2023-09-01T13:38:26.422+05:30 [Info] Using greedy index placement for index 1058987654
2023-09-01T13:38:26.424+05:30 [Info] Using greedy index placement for index 1059987654
2023-09-01T13:38:26.424+05:30 [Info] Actual variance of deferred index count across nodes is 648
--- PASS: TestGreedyPlanner (0.47s)
=== RUN   TestTenantAwarePlanner
2023/09/01 13:38:26 In TestTenantAwarePlanner()
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 1 empty node - 1 SG
2023-09-01T13:38:26.427+05:30 [Info] Planner::executeTenantAwarePlan Found SubClusters  [[127.0.0.1:9001]]
2023-09-01T13:38:26.427+05:30 [Info] Planner::executeTenantAwarePlan Found Candidate Based on Tenant Affinity []
2023-09-01T13:38:26.427+05:30 [Info] Planner::executeTenantAwarePlan Found Result [127.0.0.1:9001]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 4 empty nodes - 2 SG
2023-09-01T13:38:26.429+05:30 [Info] Planner::executeTenantAwarePlan Found SubClusters  [[127.0.0.1:9004 127.0.0.1:9001] [127.0.0.1:9003 127.0.0.1:9002]]
2023-09-01T13:38:26.429+05:30 [Info] Planner::executeTenantAwarePlan Found Candidate Based on Tenant Affinity []
2023-09-01T13:38:26.429+05:30 [Info] Planner::executeTenantAwarePlan Found Result [127.0.0.1:9003 127.0.0.1:9002]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 1 node - 1 SG
2023-09-01T13:38:26.431+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.431+05:30 [Info] Planner::executeTenantAwarePlan Found SubClusters  [[127.0.0.1:9001]]
2023-09-01T13:38:26.431+05:30 [Info] Planner::executeTenantAwarePlan Found Candidate Based on Tenant Affinity [127.0.0.1:9001]
2023-09-01T13:38:26.431+05:30 [Info] Planner::executeTenantAwarePlan Found Result [127.0.0.1:9001]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 2 nodes - 1 SG
2023-09-01T13:38:26.433+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.434+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg1 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.434+05:30 [Info] Planner::executeTenantAwarePlan Found SubClusters  [[127.0.0.1:9001] [127.0.0.1:9002]]
2023-09-01T13:38:26.434+05:30 [Info] Planner::executeTenantAwarePlan Found Candidate Based on Tenant Affinity [127.0.0.1:9001]
2023-09-01T13:38:26.434+05:30 [Info] Planner::executeTenantAwarePlan Found Result [127.0.0.1:9001]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 4 nodes - 2 SG - Tenant Affinity(a)
2023-09-01T13:38:26.436+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.436+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.436+05:30 [Info] Planner::executeTenantAwarePlan Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9002] [127.0.0.1:9004 127.0.0.1:9003]]
2023-09-01T13:38:26.436+05:30 [Info] Planner::executeTenantAwarePlan Found Candidate Based on Tenant Affinity [127.0.0.1:9001 127.0.0.1:9002]
2023-09-01T13:38:26.436+05:30 [Info] Planner::executeTenantAwarePlan Found Result [127.0.0.1:9001 127.0.0.1:9002]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 4 nodes - 2 SG - Tenant Affinity(b)
2023-09-01T13:38:26.438+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.438+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg3 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.438+05:30 [Info] Planner::executeTenantAwarePlan Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9003 127.0.0.1:9002]]
2023-09-01T13:38:26.438+05:30 [Info] Planner::executeTenantAwarePlan Found Candidate Based on Tenant Affinity [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.438+05:30 [Info] Planner::executeTenantAwarePlan Found Result [127.0.0.1:9001 127.0.0.1:9004]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 4 nodes - 2 SG - Tenant Affinity(c)
2023-09-01T13:38:26.440+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.440+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.440+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.440+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg3 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.440+05:30 [Info] Planner::executeTenantAwarePlan Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9003]]
2023-09-01T13:38:26.440+05:30 [Info] Planner::executeTenantAwarePlan Found Candidate Based on Tenant Affinity [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.440+05:30 [Info] Planner::executeTenantAwarePlan Found Result [127.0.0.1:9001 127.0.0.1:9004]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 6 nodes - 3 SG - Tenant Affinity Memory Above LWM
2023-09-01T13:38:26.442+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.442+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 700000000 Units 1000 Tenants 0
2023-09-01T13:38:26.442+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.442+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 700000000 Units 1000 Tenants 0
2023-09-01T13:38:26.442+05:30 [Info] Planner::executeTenantAwarePlan Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.442+05:30 [Info] Planner::executeTenantAwarePlan Found Candidate Based on Tenant Affinity [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.442+05:30 [Info] Planner::executeTenantAwarePlan Found Result [127.0.0.1:9002 127.0.0.1:9005]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 6 nodes - 3 SG - Tenant Affinity Units Above LWM
2023-09-01T13:38:26.444+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.444+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 300000000 Units 5000 Tenants 0
2023-09-01T13:38:26.444+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.444+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 300000000 Units 5000 Tenants 0
2023-09-01T13:38:26.444+05:30 [Info] Planner::executeTenantAwarePlan Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.444+05:30 [Info] Planner::executeTenantAwarePlan Found Candidate Based on Tenant Affinity [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.444+05:30 [Info] Planner::executeTenantAwarePlan Found Result [127.0.0.1:9002 127.0.0.1:9005]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 6 nodes - 3 SG - Tenant Affinity New Tenant(a)
2023-09-01T13:38:26.446+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.447+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 700000000 Units 1000 Tenants 0
2023-09-01T13:38:26.447+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.447+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 700000000 Units 1000 Tenants 0
2023-09-01T13:38:26.447+05:30 [Info] Planner::executeTenantAwarePlan Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.447+05:30 [Info] Planner::executeTenantAwarePlan Found Candidate Based on Tenant Affinity []
2023-09-01T13:38:26.447+05:30 [Info] Planner::executeTenantAwarePlan Found Result [127.0.0.1:9006 127.0.0.1:9003]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 6 nodes - 3 SG - Tenant Affinity New Tenant(b)
2023-09-01T13:38:26.449+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.449+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 700000000 Units 1000 Tenants 0
2023-09-01T13:38:26.449+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.449+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 700000000 Units 1000 Tenants 0
2023-09-01T13:38:26.449+05:30 [Info] Planner::executeTenantAwarePlan Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.449+05:30 [Info] Planner::executeTenantAwarePlan Found Candidate Based on Tenant Affinity []
2023-09-01T13:38:26.449+05:30 [Info] Planner::executeTenantAwarePlan Found Result [127.0.0.1:9006 127.0.0.1:9003]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 4 nodes - 2 SG - Tenant Affinity Above Units HWM
2023-09-01T13:38:26.451+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 300000000 Units 8000 Tenants 0
2023-09-01T13:38:26.451+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.451+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.451+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg3 Memory 300000000 Units 8000 Tenants 0
2023-09-01T13:38:26.451+05:30 [Info] Planner::executeTenantAwarePlan Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9003]]
2023-09-01T13:38:26.451+05:30 [Info] Planner::executeTenantAwarePlan Found Candidate Based on Tenant Affinity [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.451+05:30 [Info] Planner::executeTenantAwarePlan Found Result [127.0.0.1:9001 127.0.0.1:9004]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 6 nodes - 3 SG - New Tenant Units Above LWM
2023-09-01T13:38:26.453+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 300000000 Units 5000 Tenants 0
2023-09-01T13:38:26.453+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 300000000 Units 5500 Tenants 0
2023-09-01T13:38:26.453+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 300000000 Units 5000 Tenants 0
2023-09-01T13:38:26.453+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 300000000 Units 5500 Tenants 0
2023-09-01T13:38:26.453+05:30 [Info] Planner::executeTenantAwarePlan Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.453+05:30 [Info] Planner::executeTenantAwarePlan Found Candidate Based on Tenant Affinity []
2023-09-01T13:38:26.453+05:30 [Info] Planner::executeTenantAwarePlan Found Result [127.0.0.1:9006 127.0.0.1:9003]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 4 nodes, rebalance in progress(inst version) 
2023-09-01T13:38:26.455+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 50000000 Units 500 Tenants 0
2023-09-01T13:38:26.455+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 50000000 Units 500 Tenants 0
2023-09-01T13:38:26.455+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.455+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip index 1111:1212:0 node 127.0.0.1:9003 from subCluster grouping. Rebalance in progress.
2023-09-01T13:38:26.455+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.455+05:30 [Info] Planner::executeTenantAwarePlan Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9002] [127.0.0.1:9003 127.0.0.1:9004]]
2023-09-01T13:38:26.455+05:30 [Info] Planner::executeTenantAwarePlan Found Candidate Based on Tenant Affinity []
2023-09-01T13:38:26.455+05:30 [Info] Planner::executeTenantAwarePlan Found Result [127.0.0.1:9001 127.0.0.1:9002]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 4 nodes, rebalance in progress(rstate) 
2023-09-01T13:38:26.457+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.458+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.458+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 50000000 Units 500 Tenants 0
2023-09-01T13:38:26.458+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 50000000 Units 500 Tenants 0
2023-09-01T13:38:26.458+05:30 [Info] Planner::executeTenantAwarePlan Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9002] [127.0.0.1:9003 127.0.0.1:9004]]
2023-09-01T13:38:26.458+05:30 [Info] Planner::executeTenantAwarePlan Found Candidate Based on Tenant Affinity []
2023-09-01T13:38:26.458+05:30 [Info] Planner::executeTenantAwarePlan Found Result [127.0.0.1:9003 127.0.0.1:9004]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 4 nodes, rebalance in progress(rstate+inst version) 
2023-09-01T13:38:26.460+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.460+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.460+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 50000000 Units 500 Tenants 0
2023-09-01T13:38:26.460+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip index 1111:1212:0 node 127.0.0.1:9003 from subCluster grouping. Rebalance in progress.
2023-09-01T13:38:26.460+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 50000000 Units 500 Tenants 0
2023-09-01T13:38:26.460+05:30 [Info] Planner::executeTenantAwarePlan Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9002] [127.0.0.1:9003 127.0.0.1:9004]]
2023-09-01T13:38:26.460+05:30 [Info] Planner::executeTenantAwarePlan Found Candidate Based on Tenant Affinity []
2023-09-01T13:38:26.460+05:30 [Info] Planner::executeTenantAwarePlan Found Result [127.0.0.1:9003 127.0.0.1:9004]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 4 nodes, rebalance pending cleanup
2023-09-01T13:38:26.462+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.462+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.462+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 50000000 Units 500 Tenants 0
2023-09-01T13:38:26.462+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 50000000 Units 500 Tenants 0
2023-09-01T13:38:26.462+05:30 [Info] Planner::executeTenantAwarePlan Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9002] [127.0.0.1:9003 127.0.0.1:9004]]
2023-09-01T13:38:26.462+05:30 [Info] Planner::executeTenantAwarePlan Found Candidate Based on Tenant Affinity []
2023-09-01T13:38:26.462+05:30 [Info] Planner::executeTenantAwarePlan Found Result [127.0.0.1:9003 127.0.0.1:9004]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 2 empty nodes - 1 SG
2023-09-01T13:38:26.464+05:30 [Info] Planner::executeTenantAwarePlan Found SubClusters  [[127.0.0.1:9001] [127.0.0.1:9002]]
2023-09-01T13:38:26.464+05:30 [Info] Planner::executeTenantAwarePlan Found Candidate Based on Tenant Affinity []
2023-09-01T13:38:26.464+05:30 [Info] Planner::filterPartialSubClusters Filter partial subcluster [127.0.0.1:9002]
2023-09-01T13:38:26.464+05:30 [Info] Planner::filterPartialSubClusters Filter partial subcluster [127.0.0.1:9001]
2023-09-01T13:38:26.464+05:30 [Info] Planner::executeTenantAwarePlan Found no matching candidate for tenant &{idx1 bucket1   987654 false [name1]  false false false false 0  0 [] 2 []   false 0 0 0 0 0 0 0 0}. Error - Unable to find any valid SubCluster
2023/09/01 13:38:26 Expected error Planner not able to find any node for placement - Unable to find any valid SubCluster
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 4 nodes - 2 SG - Tenant Affinity Above Memory HWM
2023-09-01T13:38:26.464+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 1000 Tenants 0
2023-09-01T13:38:26.464+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.464+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.464+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg3 Memory 900000000 Units 1000 Tenants 0
2023-09-01T13:38:26.464+05:30 [Info] Planner::executeTenantAwarePlan Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9003]]
2023-09-01T13:38:26.464+05:30 [Info] Planner::executeTenantAwarePlan Found Candidate Based on Tenant Affinity [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.464+05:30 [Info] Planner::executeTenantAwarePlan Found no matching candidate for tenant &{idx1 bucket1   987654 false [name1]  false false false false 0  0 [] 2 []   false 0 0 0 0 0 0 0 0}. Error - Tenant SubCluster Above High Usage Threshold
2023/09/01 13:38:26 Expected error Planner not able to find any node for placement - Tenant SubCluster Above High Usage Threshold
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 6 nodes - 3 SG - New Tenant Memory Above LWM
2023-09-01T13:38:26.465+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 680000000 Units 1000 Tenants 0
2023-09-01T13:38:26.465+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 700000000 Units 1000 Tenants 0
2023-09-01T13:38:26.465+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 680000000 Units 1000 Tenants 0
2023-09-01T13:38:26.465+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 700000000 Units 1000 Tenants 0
2023-09-01T13:38:26.465+05:30 [Info] Planner::executeTenantAwarePlan Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.465+05:30 [Info] Planner::executeTenantAwarePlan Found Candidate Based on Tenant Affinity []
2023-09-01T13:38:26.465+05:30 [Info] Planner::executeTenantAwarePlan Found no matching candidate for tenant &{idx1 bucket7   987654 false [name1]  false false false false 0  0 [] 2 []   false 0 0 0 0 0 0 0 0}. Error - No SubCluster Below Low Usage Threshold
2023/09/01 13:38:26 Expected error Planner not able to find any node for placement - No SubCluster Below Low Usage Threshold
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 6 nodes - 3 SG - New Tenant Units Above LWM
2023-09-01T13:38:26.465+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 300000000 Units 5000 Tenants 0
2023-09-01T13:38:26.466+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 300000000 Units 5500 Tenants 0
2023-09-01T13:38:26.466+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 0 Units 4500 Tenants 0
2023-09-01T13:38:26.466+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 300000000 Units 5000 Tenants 0
2023-09-01T13:38:26.466+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 300000000 Units 5500 Tenants 0
2023-09-01T13:38:26.466+05:30 [Error] Planner::validateSubClusterGrouping  Node 127.0.0.1:9001 belongs to multiple subclusters [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9001 127.0.0.1:9003 127.0.0.1:9004]].
2023/09/01 13:38:26 Expected error Planner Constraint Violation. Node 127.0.0.1:9001 belongs to multiple subclusters [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9001 127.0.0.1:9003 127.0.0.1:9004]].
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 4 nodes, rebalance in progress(inst version) 
2023-09-01T13:38:26.466+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.466+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.466+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 50000000 Units 500 Tenants 0
2023-09-01T13:38:26.466+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip index 1111:1212:0 node 127.0.0.1:9003 from subCluster grouping. Rebalance in progress.
2023-09-01T13:38:26.466+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Cannot group node 127.0.0.1:9003 due to in progress rebalance for all buckets on the node. map[bucket1:true]
2023/09/01 13:38:26 Expected error Rebalance in progress or cleanup pending from previous rebalance.
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 4 nodes, rebalance in progress(rstate) 
2023-09-01T13:38:26.467+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.467+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.467+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Cannot group empty node 127.0.0.1:9003 due to in progress rebalance for all buckets on the node. map[bucket1:true]
2023/09/01 13:38:26 Expected error Rebalance in progress or cleanup pending from previous rebalance.
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 4 nodes, rebalance in progress(rstate+inst version) 
2023-09-01T13:38:26.467+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.467+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.467+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 50000000 Units 500 Tenants 0
2023-09-01T13:38:26.467+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip index 1111:1212:0 node 127.0.0.1:9003 from subCluster grouping. Rebalance in progress.
2023-09-01T13:38:26.467+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Cannot group node 127.0.0.1:9003 due to in progress rebalance for all buckets on the node. map[bucket1:true]
2023/09/01 13:38:26 Expected error Rebalance in progress or cleanup pending from previous rebalance.
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 4 nodes, rebalance pending cleanup
2023-09-01T13:38:26.468+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.468+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.468+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Cannot group empty node 127.0.0.1:9004 due to in progress rebalance for all buckets on the node. map[bucket1:true]
2023/09/01 13:38:26 Expected error Rebalance in progress or cleanup pending from previous rebalance.
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 3 SG, 1 empty, 1 Memory Above HWM
2023-09-01T13:38:26.469+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.469+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 900000000 Units 1000 Tenants 0
2023-09-01T13:38:26.469+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.469+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 900000000 Units 1000 Tenants 0
2023-09-01T13:38:26.469+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.469+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [[127.0.0.1:9002 127.0.0.1:9005]]
2023-09-01T13:38:26.469+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.469+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket3 MemoryUsage 400000000 UnitsUsage 500 
2023-09-01T13:38:26.469+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters Below LWM [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.469+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket3 MemoryUsage 400000000 UnitsUsage 500  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 3 SG, 1 empty, 1 Units Above HWM
2023-09-01T13:38:26.470+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 400000000 Units 8000 Tenants 0
2023-09-01T13:38:26.470+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 400000000 Units 1000 Tenants 0
2023-09-01T13:38:26.470+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 400000000 Units 8000 Tenants 0
2023-09-01T13:38:26.470+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 400000000 Units 1000 Tenants 0
2023-09-01T13:38:26.470+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.470+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [[127.0.0.1:9001 127.0.0.1:9004]]
2023-09-01T13:38:26.470+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.470+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 200000000 UnitsUsage 3000 
2023-09-01T13:38:26.470+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters Below LWM [[127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.470+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 200000000 UnitsUsage 3000  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 3 SG, 1 empty, Both Memory/Units Above HWM
2023-09-01T13:38:26.471+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 9000 Tenants 0
2023-09-01T13:38:26.471+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.471+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 9000 Tenants 0
2023-09-01T13:38:26.471+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.471+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.471+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [[127.0.0.1:9001 127.0.0.1:9004]]
2023-09-01T13:38:26.471+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.471+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 300000000 UnitsUsage 3000 
2023-09-01T13:38:26.471+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 200000000 UnitsUsage 2000 
2023-09-01T13:38:26.471+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters Below LWM [[127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.472+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 300000000 UnitsUsage 3000  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.472+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 200000000 UnitsUsage 2000  can be placed on [127.0.0.1:9002 127.0.0.1:9005]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 3 SG, Multiple tenants to move, single source, multiple destination
2023-09-01T13:38:26.472+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 9000 Tenants 0
2023-09-01T13:38:26.472+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 200000000 Units 1000 Tenants 0
2023-09-01T13:38:26.472+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 9000 Tenants 0
2023-09-01T13:38:26.472+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 200000000 Units 1000 Tenants 0
2023-09-01T13:38:26.472+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.472+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [[127.0.0.1:9001 127.0.0.1:9004]]
2023-09-01T13:38:26.472+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.472+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 200000000 UnitsUsage 4000 
2023-09-01T13:38:26.473+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket5 MemoryUsage 400000000 UnitsUsage 2000 
2023-09-01T13:38:26.473+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters Below LWM [[127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.473+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 200000000 UnitsUsage 4000  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.473+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket5 MemoryUsage 400000000 UnitsUsage 2000  can be placed on [127.0.0.1:9002 127.0.0.1:9005]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 3 SG, Multiple tenants to move, no nodes below LWM
2023-09-01T13:38:26.473+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 300000000 Units 5000 Tenants 0
2023-09-01T13:38:26.473+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 900000000 Units 3000 Tenants 0
2023-09-01T13:38:26.473+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 700000000 Units 3000 Tenants 0
2023-09-01T13:38:26.473+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 300000000 Units 5000 Tenants 0
2023-09-01T13:38:26.473+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 900000000 Units 3000 Tenants 0
2023-09-01T13:38:26.473+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 700000000 Units 3000 Tenants 0
2023-09-01T13:38:26.474+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9003 127.0.0.1:9006]]
2023-09-01T13:38:26.474+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [[127.0.0.1:9002 127.0.0.1:9005]]
2023-09-01T13:38:26.474+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.474+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket3 MemoryUsage 300000000 UnitsUsage 500 
2023-09-01T13:38:26.474+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket1 MemoryUsage 200000000 UnitsUsage 500 
2023-09-01T13:38:26.474+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters Below LWM []
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 4 SG, Multiple tenants to move, multiple source, multiple destination(non-uniform memory/units usage)
2023-09-01T13:38:26.475+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.475+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.475+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.475+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.475+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9007 SG sg1 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.475+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.475+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9007 127.0.0.1:9008] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.475+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005]]
2023-09-01T13:38:26.475+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.475+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 70000000 UnitsUsage 1200 
2023-09-01T13:38:26.475+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket3 MemoryUsage 80000000 UnitsUsage 800 
2023-09-01T13:38:26.475+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket1 MemoryUsage 30000000 UnitsUsage 1500 
2023-09-01T13:38:26.475+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 120000000 UnitsUsage 500 
2023-09-01T13:38:26.475+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.475+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket9 MemoryUsage 100000000 UnitsUsage 500 
2023-09-01T13:38:26.475+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket10 MemoryUsage 200000000 UnitsUsage 800 
2023-09-01T13:38:26.475+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket8 MemoryUsage 75000000 UnitsUsage 700 
2023-09-01T13:38:26.475+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket7 MemoryUsage 25000000 UnitsUsage 1000 
2023-09-01T13:38:26.475+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters Below LWM [[127.0.0.1:9007 127.0.0.1:9008] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.475+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 70000000 UnitsUsage 1200  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.475+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket9 MemoryUsage 100000000 UnitsUsage 500  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.475+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket3 MemoryUsage 80000000 UnitsUsage 800  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.475+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket10 MemoryUsage 200000000 UnitsUsage 800  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.475+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket1 MemoryUsage 30000000 UnitsUsage 1500  can be placed on [127.0.0.1:9007 127.0.0.1:9008]
2023-09-01T13:38:26.475+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket8 MemoryUsage 75000000 UnitsUsage 700  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.475+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 120000000 UnitsUsage 500  can be placed on [127.0.0.1:9007 127.0.0.1:9008]
2023-09-01T13:38:26.475+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket7 MemoryUsage 25000000 UnitsUsage 1000  can be placed on [127.0.0.1:9007 127.0.0.1:9008]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 4 SG, Multiple tenants to move, multiple source, multiple destination(non-uniform memory/units usage)
2023-09-01T13:38:26.476+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.476+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.476+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.476+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.476+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9007 SG sg1 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.476+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.476+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9007 127.0.0.1:9008] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.476+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005]]
2023-09-01T13:38:26.476+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.476+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 120000000 UnitsUsage 1500 
2023-09-01T13:38:26.476+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket3 MemoryUsage 80000000 UnitsUsage 1200 
2023-09-01T13:38:26.476+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 70000000 UnitsUsage 800 
2023-09-01T13:38:26.476+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket1 MemoryUsage 30000000 UnitsUsage 500 
2023-09-01T13:38:26.476+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.476+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket9 MemoryUsage 100000000 UnitsUsage 700 
2023-09-01T13:38:26.476+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket10 MemoryUsage 200000000 UnitsUsage 800 
2023-09-01T13:38:26.476+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket8 MemoryUsage 75000000 UnitsUsage 1000 
2023-09-01T13:38:26.476+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket7 MemoryUsage 25000000 UnitsUsage 500 
2023-09-01T13:38:26.476+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters Below LWM [[127.0.0.1:9007 127.0.0.1:9008] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.476+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 120000000 UnitsUsage 1500  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.476+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket9 MemoryUsage 100000000 UnitsUsage 700  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.476+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket3 MemoryUsage 80000000 UnitsUsage 1200  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.476+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket10 MemoryUsage 200000000 UnitsUsage 800  can be placed on [127.0.0.1:9007 127.0.0.1:9008]
2023-09-01T13:38:26.476+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 70000000 UnitsUsage 800  can be placed on [127.0.0.1:9007 127.0.0.1:9008]
2023-09-01T13:38:26.476+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket8 MemoryUsage 75000000 UnitsUsage 1000  can be placed on [127.0.0.1:9007 127.0.0.1:9008]
2023-09-01T13:38:26.476+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket1 MemoryUsage 30000000 UnitsUsage 500  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.476+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket7 MemoryUsage 25000000 UnitsUsage 500  can be placed on [127.0.0.1:9007 127.0.0.1:9008]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 3 SG, Single Large Tenant, Nothing to move
2023-09-01T13:38:26.477+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 1000 Tenants 0
2023-09-01T13:38:26.477+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 900000000 Units 2000 Tenants 0
2023-09-01T13:38:26.477+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 1000 Tenants 0
2023-09-01T13:38:26.477+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 900000000 Units 2000 Tenants 0
2023-09-01T13:38:26.477+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.477+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005]]
2023-09-01T13:38:26.477+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.477+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket3 MemoryUsage 200000000 UnitsUsage 1000 
2023-09-01T13:38:26.477+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters Below LWM [[127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.477+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket3 MemoryUsage 200000000 UnitsUsage 1000  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 4 SG, Multiple tenants to move, multiple source, multiple destination(zero usage tenants)
2023-09-01T13:38:26.479+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.479+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.479+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.479+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.479+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9007 SG sg1 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.479+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.479+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9007 127.0.0.1:9008] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.479+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005]]
2023-09-01T13:38:26.479+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.479+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 120000000 UnitsUsage 1500 
2023-09-01T13:38:26.479+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket3 MemoryUsage 80000000 UnitsUsage 1200 
2023-09-01T13:38:26.479+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 70000000 UnitsUsage 800 
2023-09-01T13:38:26.479+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket1 MemoryUsage 30000000 UnitsUsage 500 
2023-09-01T13:38:26.479+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket17 MemoryUsage 0 UnitsUsage 0 
2023-09-01T13:38:26.479+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket14 MemoryUsage 0 UnitsUsage 0 
2023-09-01T13:38:26.479+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.479+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket9 MemoryUsage 100000000 UnitsUsage 700 
2023-09-01T13:38:26.479+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket10 MemoryUsage 200000000 UnitsUsage 800 
2023-09-01T13:38:26.479+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket8 MemoryUsage 75000000 UnitsUsage 1000 
2023-09-01T13:38:26.479+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket7 MemoryUsage 25000000 UnitsUsage 500 
2023-09-01T13:38:26.479+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket15 MemoryUsage 0 UnitsUsage 0 
2023-09-01T13:38:26.479+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket16 MemoryUsage 0 UnitsUsage 0 
2023-09-01T13:38:26.479+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters Below LWM [[127.0.0.1:9007 127.0.0.1:9008] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.479+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 120000000 UnitsUsage 1500  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.479+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket9 MemoryUsage 100000000 UnitsUsage 700  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.479+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket3 MemoryUsage 80000000 UnitsUsage 1200  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.479+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket10 MemoryUsage 200000000 UnitsUsage 800  can be placed on [127.0.0.1:9007 127.0.0.1:9008]
2023-09-01T13:38:26.479+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 70000000 UnitsUsage 800  can be placed on [127.0.0.1:9007 127.0.0.1:9008]
2023-09-01T13:38:26.479+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket8 MemoryUsage 75000000 UnitsUsage 1000  can be placed on [127.0.0.1:9007 127.0.0.1:9008]
2023-09-01T13:38:26.479+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket1 MemoryUsage 30000000 UnitsUsage 500  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.480+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket7 MemoryUsage 25000000 UnitsUsage 500  can be placed on [127.0.0.1:9007 127.0.0.1:9008]
2023-09-01T13:38:26.480+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket17 MemoryUsage 0 UnitsUsage 0  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.480+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket15 MemoryUsage 0 UnitsUsage 0  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.480+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket14 MemoryUsage 0 UnitsUsage 0  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.480+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket16 MemoryUsage 0 UnitsUsage 0  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 3 SG, 1 Partial Subcluster
2023-09-01T13:38:26.481+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.481+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 900000000 Units 3000 Tenants 0
2023-09-01T13:38:26.481+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.481+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 900000000 Units 3000 Tenants 0
2023-09-01T13:38:26.481+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9003]]
2023-09-01T13:38:26.481+05:30 [Info] Planner::repairMissingReplica Found SubCluster [127.0.0.1:9003] with len 1. Skipping replica repair attempt.
2023-09-01T13:38:26.481+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [[127.0.0.1:9002 127.0.0.1:9005]]
2023-09-01T13:38:26.481+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.481+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket3 MemoryUsage 300000000 UnitsUsage 500 
2023-09-01T13:38:26.481+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket1 MemoryUsage 200000000 UnitsUsage 500 
2023-09-01T13:38:26.481+05:30 [Info] Planner::filterPartialSubClusters Filter partial subcluster [127.0.0.1:9003]
2023-09-01T13:38:26.481+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters Below LWM [[127.0.0.1:9001 127.0.0.1:9004]]
2023-09-01T13:38:26.481+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket3 MemoryUsage 300000000 UnitsUsage 500  can be placed on [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.481+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket1 MemoryUsage 200000000 UnitsUsage 500  can be placed on [127.0.0.1:9001 127.0.0.1:9004]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Replica Repair - 4 SG, Missing Replicas for multiple tenants in SG
2023-09-01T13:38:26.482+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.482+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.482+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.482+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.482+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9007 SG sg1 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.482+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.482+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9007 127.0.0.1:9008] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.482+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket2,,,idx2,1) on 127.0.0.1:9004
2023-09-01T13:38:26.482+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket4,,,idx2,0) on 127.0.0.1:9001
2023-09-01T13:38:26.482+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket8,,,idx1,1) on 127.0.0.1:9005
2023-09-01T13:38:26.482+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket8,,,idx1,1) on 127.0.0.1:9005
2023-09-01T13:38:26.482+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM []
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Replica Repair - 4 SG, Missing Replicas, Buddy Node Failed over
2023-09-01T13:38:26.483+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.483+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.483+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.483+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9007 SG sg1 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.483+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.483+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9007 127.0.0.1:9008] [127.0.0.1:9003]]
2023-09-01T13:38:26.483+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket1,,,idx1,1) on 127.0.0.1:9004
2023-09-01T13:38:26.483+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket2,,,idx2,1) on 127.0.0.1:9004
2023-09-01T13:38:26.483+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket3,,,idx1,1) on 127.0.0.1:9004
2023-09-01T13:38:26.483+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket8,,,idx1,1) on 127.0.0.1:9005
2023-09-01T13:38:26.483+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket8,,,idx1,1) on 127.0.0.1:9005
2023-09-01T13:38:26.483+05:30 [Info] Planner::repairMissingReplica Found SubCluster [127.0.0.1:9003] with len 1. Skipping replica repair attempt.
2023-09-01T13:38:26.484+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM []
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Replica Repair - 4 SG, Missing Replicas, Buddy Node Failed over, No replacement
2023-09-01T13:38:26.484+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.484+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.484+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.484+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9007 SG sg1 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.484+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.484+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9007 127.0.0.1:9008]]
2023-09-01T13:38:26.484+05:30 [Info] Planner::repairMissingReplica Found SubCluster [127.0.0.1:9001] with len 1. Skipping replica repair attempt.
2023-09-01T13:38:26.484+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM []
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Replica Repair - 4 SG, Missing Replicas, one replica missing with pendingDelete true 
2023-09-01T13:38:26.485+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.485+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.485+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.485+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.485+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9007 SG sg1 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.485+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.485+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9007 127.0.0.1:9008] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.485+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket2,,,idx2,1) on 127.0.0.1:9004
2023-09-01T13:38:26.485+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket4,,,idx2,0) on 127.0.0.1:9001
2023-09-01T13:38:26.485+05:30 [Info] Planner::findMissingReplicaForIndexerNode Skipping Replica Repair for 88883:81813:0. PendingDelete true
2023-09-01T13:38:26.485+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket8,,,idx2,1) on 127.0.0.1:9005
2023-09-01T13:38:26.485+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM []
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Replica Repair - 2 SG, Missing Replicas with Nodes over HWM
2023-09-01T13:38:26.486+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 7000 Tenants 0
2023-09-01T13:38:26.487+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 100000000 Units 500 Tenants 0
2023-09-01T13:38:26.487+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 7000 Tenants 0
2023-09-01T13:38:26.487+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005]]
2023-09-01T13:38:26.487+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket9,,,idx2,1) on 127.0.0.1:9005
2023-09-01T13:38:26.487+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [[127.0.0.1:9001 127.0.0.1:9004]]
2023-09-01T13:38:26.487+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.487+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket3 MemoryUsage 80000000 UnitsUsage 800 
2023-09-01T13:38:26.487+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 70000000 UnitsUsage 700 
2023-09-01T13:38:26.487+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 120000000 UnitsUsage 500 
2023-09-01T13:38:26.487+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket1 MemoryUsage 30000000 UnitsUsage 1000 
2023-09-01T13:38:26.487+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters Below LWM [[127.0.0.1:9002 127.0.0.1:9005]]
2023-09-01T13:38:26.487+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket3 MemoryUsage 80000000 UnitsUsage 800  can be placed on [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.487+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 70000000 UnitsUsage 700  can be placed on [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.487+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 120000000 UnitsUsage 500  can be placed on [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.487+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket1 MemoryUsage 30000000 UnitsUsage 1000  can be placed on [127.0.0.1:9002 127.0.0.1:9005]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Swap Rebalance - 4 SG, Swap 1 node each from 2 SG with 2 new nodes
2023-09-01T13:38:26.488+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9002 127.0.0.1:9004]
2023-09-01T13:38:26.488+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9002 127.0.0.1:9004]
2023-09-01T13:38:26.488+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9005 127.0.0.1:9001]
2023-09-01T13:38:26.488+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes [127.0.0.1:9003 127.0.0.1:9006 127.0.0.1:9007 127.0.0.1:9008]
2023-09-01T13:38:26.488+05:30 [Info] Planner::moveTenantsFromDeletedNodes selected newNodes for swap [127.0.0.1:9003 127.0.0.1:9008 127.0.0.1:9006 127.0.0.1:9007]
2023-09-01T13:38:26.488+05:30 [Info] Moving index 7777:7171:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003
2023-09-01T13:38:26.488+05:30 [Info] Moving index 8888:8181:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003
2023-09-01T13:38:26.488+05:30 [Info] Moving index 9999:9191:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003
2023-09-01T13:38:26.488+05:30 [Info] Moving index 101010:101101:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003
2023-09-01T13:38:26.488+05:30 [Info] Moving index 111111:112112:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003
2023-09-01T13:38:26.488+05:30 [Info] Moving index 121212:121121:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003
2023-09-01T13:38:26.488+05:30 [Info] Moving index 1111:11212:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9008
2023-09-01T13:38:26.488+05:30 [Info] Moving index 2222:22121:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9008
2023-09-01T13:38:26.488+05:30 [Info] Moving index 3333:33131:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9008
2023-09-01T13:38:26.488+05:30 [Info] Moving index 4444:44141:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9008
2023-09-01T13:38:26.488+05:30 [Info] Moving index 5555:55151:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9008
2023-09-01T13:38:26.488+05:30 [Info] Moving index 6666:66161:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9008
2023-09-01T13:38:26.488+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9002 SG sg2 Memory 0 Units 0
2023-09-01T13:38:26.489+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9004 SG sg2 Memory 0 Units 0
2023-09-01T13:38:26.489+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.489+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.489+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.489+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg3 Memory 1100000000 Units 8100 Tenants 0
2023-09-01T13:38:26.489+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9008] [127.0.0.1:9003 127.0.0.1:9005] [127.0.0.1:9006] [127.0.0.1:9007]]
2023-09-01T13:38:26.489+05:30 [Info] Planner::repairMissingReplica Found SubCluster [127.0.0.1:9006] with len 1. Skipping replica repair attempt.
2023-09-01T13:38:26.489+05:30 [Info] Planner::repairMissingReplica Found SubCluster [127.0.0.1:9007] with len 1. Skipping replica repair attempt.
2023-09-01T13:38:26.489+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM []
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Swap Rebalance - 4 SG, Swap 1 node each from 2 SG with 2 new nodes(different SG)
2023-09-01T13:38:26.490+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9004 127.0.0.1:9005]
2023-09-01T13:38:26.490+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9004 127.0.0.1:9005]
2023-09-01T13:38:26.490+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9001 127.0.0.1:9002]
2023-09-01T13:38:26.490+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.490+05:30 [Info] Planner::moveTenantsFromDeletedNodes selected newNodes for swap [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.490+05:30 [Info] Moving index 1111:11212:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9006
2023-09-01T13:38:26.490+05:30 [Info] Moving index 2222:22121:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9006
2023-09-01T13:38:26.490+05:30 [Info] Moving index 3333:33131:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9006
2023-09-01T13:38:26.490+05:30 [Info] Moving index 4444:44141:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9006
2023-09-01T13:38:26.490+05:30 [Info] Moving index 5555:55151:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9006
2023-09-01T13:38:26.490+05:30 [Info] Moving index 6666:66161:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9006
2023-09-01T13:38:26.490+05:30 [Info] Moving index 7777:17171:0 from source 127.0.0.1:9005 to dest 127.0.0.1:9003
2023-09-01T13:38:26.490+05:30 [Info] Moving index 8888:18181:0 from source 127.0.0.1:9005 to dest 127.0.0.1:9003
2023-09-01T13:38:26.490+05:30 [Info] Moving index 9999:19191:0 from source 127.0.0.1:9005 to dest 127.0.0.1:9003
2023-09-01T13:38:26.490+05:30 [Info] Moving index 101010:1101101:0 from source 127.0.0.1:9005 to dest 127.0.0.1:9003
2023-09-01T13:38:26.490+05:30 [Info] Moving index 111111:1112112:0 from source 127.0.0.1:9005 to dest 127.0.0.1:9003
2023-09-01T13:38:26.490+05:30 [Info] Moving index 121212:1121121:0 from source 127.0.0.1:9005 to dest 127.0.0.1:9003
2023-09-01T13:38:26.490+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9004 SG sg2 Memory 0 Units 0
2023-09-01T13:38:26.490+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9005 SG sg3 Memory 0 Units 0
2023-09-01T13:38:26.490+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.490+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.490+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.490+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.490+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9007 SG sg1 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.490+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.490+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9006] [127.0.0.1:9002 127.0.0.1:9003] [127.0.0.1:9007 127.0.0.1:9008]]
2023-09-01T13:38:26.490+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM []
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Swap Rebalance - 4 SG, Swap 1 SG with 2 new nodes
2023-09-01T13:38:26.491+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.491+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.491+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9004 127.0.0.1:9001]
2023-09-01T13:38:26.491+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.491+05:30 [Info] Planner::moveTenantsFromDeletedNodes selected newNodes for swap [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.491+05:30 [Info] Moving index 1111:1212:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9003
2023-09-01T13:38:26.491+05:30 [Info] Moving index 2222:2121:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9003
2023-09-01T13:38:26.491+05:30 [Info] Moving index 3333:3131:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9003
2023-09-01T13:38:26.491+05:30 [Info] Moving index 4444:4141:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9003
2023-09-01T13:38:26.491+05:30 [Info] Moving index 5555:5151:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9003
2023-09-01T13:38:26.491+05:30 [Info] Moving index 6666:6161:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9003
2023-09-01T13:38:26.491+05:30 [Info] Moving index 1111:11212:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9006
2023-09-01T13:38:26.491+05:30 [Info] Moving index 2222:22121:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9006
2023-09-01T13:38:26.491+05:30 [Info] Moving index 3333:33131:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9006
2023-09-01T13:38:26.492+05:30 [Info] Moving index 4444:44141:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9006
2023-09-01T13:38:26.492+05:30 [Info] Moving index 5555:55151:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9006
2023-09-01T13:38:26.492+05:30 [Info] Moving index 6666:66161:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9006
2023-09-01T13:38:26.492+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9001 SG sg1 Memory 0 Units 0
2023-09-01T13:38:26.492+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9004 SG sg2 Memory 0 Units 0
2023-09-01T13:38:26.492+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.492+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg2 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.492+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.492+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.492+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9007 SG sg1 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.492+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.492+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9003 127.0.0.1:9006] [127.0.0.1:9007 127.0.0.1:9008]]
2023-09-01T13:38:26.492+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM []
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Swap Rebalance - 4 SG, Swap 1 node with 2 new nodes
2023-09-01T13:38:26.493+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9004]
2023-09-01T13:38:26.493+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9004]
2023-09-01T13:38:26.493+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9001]
2023-09-01T13:38:26.493+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes [127.0.0.1:9003 127.0.0.1:9007 127.0.0.1:9008 127.0.0.1:9006]
2023-09-01T13:38:26.493+05:30 [Info] Planner::moveTenantsFromDeletedNodes selected newNodes for swap [127.0.0.1:9006 127.0.0.1:9003 127.0.0.1:9007 127.0.0.1:9008]
2023-09-01T13:38:26.493+05:30 [Info] Moving index 1111:11212:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9006
2023-09-01T13:38:26.493+05:30 [Info] Moving index 2222:22121:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9006
2023-09-01T13:38:26.493+05:30 [Info] Moving index 3333:33131:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9006
2023-09-01T13:38:26.493+05:30 [Info] Moving index 4444:44141:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9006
2023-09-01T13:38:26.493+05:30 [Info] Moving index 5555:55151:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9006
2023-09-01T13:38:26.493+05:30 [Info] Moving index 6666:66161:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9006
2023-09-01T13:38:26.493+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9004 SG sg2 Memory 0 Units 0
2023-09-01T13:38:26.493+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.493+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.493+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.493+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.493+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9006] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9003] [127.0.0.1:9008] [127.0.0.1:9007]]
2023-09-01T13:38:26.493+05:30 [Info] Planner::repairMissingReplica Found SubCluster [127.0.0.1:9003] with len 1. Skipping replica repair attempt.
2023-09-01T13:38:26.493+05:30 [Info] Planner::repairMissingReplica Found SubCluster [127.0.0.1:9008] with len 1. Skipping replica repair attempt.
2023-09-01T13:38:26.493+05:30 [Info] Planner::repairMissingReplica Found SubCluster [127.0.0.1:9007] with len 1. Skipping replica repair attempt.
2023-09-01T13:38:26.493+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM []
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Swap Rebalance - 4 SG, Swap 1 empty node with 2 new nodes
2023-09-01T13:38:26.494+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9004]
2023-09-01T13:38:26.494+05:30 [Info] Planner::moveTenantsFromDeletedNodes No non-empty deleted nodes found.
2023-09-01T13:38:26.494+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9004 SG sg2 Memory 900000000 Units 8000
2023-09-01T13:38:26.494+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.494+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.494+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.494+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9007 SG sg1 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.494+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.494+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9006] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9007 127.0.0.1:9008] [127.0.0.1:9003]]
2023-09-01T13:38:26.494+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket1,,,idx1,1) on 127.0.0.1:9006
2023-09-01T13:38:26.494+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket2,,,idx2,1) on 127.0.0.1:9006
2023-09-01T13:38:26.494+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket3,,,idx1,1) on 127.0.0.1:9006
2023-09-01T13:38:26.494+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket4,,,idx2,1) on 127.0.0.1:9006
2023-09-01T13:38:26.494+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket5,,,idx1,1) on 127.0.0.1:9006
2023-09-01T13:38:26.494+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket6,,,idx2,1) on 127.0.0.1:9006
2023-09-01T13:38:26.494+05:30 [Info] Planner::repairMissingReplica Found SubCluster [127.0.0.1:9003] with len 1. Skipping replica repair attempt.
2023-09-01T13:38:26.494+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM []
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Swap Rebalance - 1 SG, Swap 1 node - Failed swap rebalance
2023-09-01T13:38:26.495+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9004]
2023-09-01T13:38:26.495+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9004]
2023-09-01T13:38:26.495+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9001]
2023-09-01T13:38:26.495+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes []
2023-09-01T13:38:26.495+05:30 [Info] Planner::moveTenantsFromDeletedNodes Num deleted nodes 1 is more than num new/empty nodes 0
2023-09-01T13:38:26.495+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.495+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 8000
2023-09-01T13:38:26.495+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg3 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.495+05:30 [Info] Planner::moveTenantsFromDeletedNodes Considering 127.0.0.1:9008 as replacement node for deleted node 127.0.0.1:9004.
2023-09-01T13:38:26.496+05:30 [Info] Moving index 1111:11212:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9008
2023-09-01T13:38:26.496+05:30 [Info] Moving index 2222:22121:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9008
2023-09-01T13:38:26.496+05:30 [Info] Moving index 3333:33131:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9008
2023-09-01T13:38:26.496+05:30 [Info] Moving index 4444:44141:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9008
2023-09-01T13:38:26.496+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9004 SG sg2 Memory 600000000 Units 4000
2023-09-01T13:38:26.496+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.496+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg3 Memory 500000000 Units 4100 Tenants 0
2023-09-01T13:38:26.496+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9008]]
2023-09-01T13:38:26.496+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM []
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Swap Rebalance - 1 SG, Swap 2 node - server group mismatch
2023-09-01T13:38:26.496+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9001 127.0.0.1:9002]
2023-09-01T13:38:26.496+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9001 127.0.0.1:9002]
2023-09-01T13:38:26.496+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9002 127.0.0.1:9001]
2023-09-01T13:38:26.496+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes [127.0.0.1:9003 127.0.0.1:9004]
2023/09/01 13:38:26 Expected error Planner - Unable to satisfy server group constraint while replacing removed nodes with new nodes.
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 4 SG, Move out 1 subcluster, Enough Capacity
2023-09-01T13:38:26.498+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.498+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.498+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9004 127.0.0.1:9001]
2023-09-01T13:38:26.498+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes []
2023-09-01T13:38:26.498+05:30 [Info] Planner::moveTenantsFromDeletedNodes Num deleted nodes 2 is more than num new/empty nodes 0
2023-09-01T13:38:26.498+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantsToBeMoved from source [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.498+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket5 MemoryUsage 350000000 UnitsUsage 2000 
2023-09-01T13:38:26.498+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket6 MemoryUsage 250000000 UnitsUsage 2000 
2023-09-01T13:38:26.498+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 120000000 UnitsUsage 1500 
2023-09-01T13:38:26.498+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket3 MemoryUsage 80000000 UnitsUsage 1200 
2023-09-01T13:38:26.498+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 70000000 UnitsUsage 800 
2023-09-01T13:38:26.498+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket1 MemoryUsage 30000000 UnitsUsage 500 
2023-09-01T13:38:26.498+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket17 MemoryUsage 0 UnitsUsage 0 
2023-09-01T13:38:26.498+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket14 MemoryUsage 0 UnitsUsage 0 
2023-09-01T13:38:26.498+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 8000
2023-09-01T13:38:26.498+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.498+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 100000000 Units 500 Tenants 0
2023-09-01T13:38:26.498+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 8000
2023-09-01T13:38:26.498+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.498+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 100000000 Units 500 Tenants 0
2023-09-01T13:38:26.498+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9007 SG sg1 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.498+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.498+05:30 [Info] Planner::moveTenantsFromDeletedNodes Found SubClusters Below LWM [[127.0.0.1:9003 127.0.0.1:9006] [127.0.0.1:9007 127.0.0.1:9008]]
2023-09-01T13:38:26.498+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket5 MemoryUsage 350000000 UnitsUsage 2000  can be placed on [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.498+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket6 MemoryUsage 250000000 UnitsUsage 2000  can be placed on [127.0.0.1:9007 127.0.0.1:9008]
2023-09-01T13:38:26.498+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 120000000 UnitsUsage 1500  can be placed on [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.498+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket3 MemoryUsage 80000000 UnitsUsage 1200  can be placed on [127.0.0.1:9007 127.0.0.1:9008]
2023-09-01T13:38:26.498+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 70000000 UnitsUsage 800  can be placed on [127.0.0.1:9007 127.0.0.1:9008]
2023-09-01T13:38:26.498+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket1 MemoryUsage 30000000 UnitsUsage 500  can be placed on [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.498+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket17 MemoryUsage 0 UnitsUsage 0  can be placed on [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.499+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket14 MemoryUsage 0 UnitsUsage 0  can be placed on [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.499+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9001 SG sg1 Memory 0 Units 0
2023-09-01T13:38:26.499+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9004 SG sg2 Memory 0 Units 0
2023-09-01T13:38:26.499+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.499+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 600000000 Units 4500 Tenants 0
2023-09-01T13:38:26.499+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.499+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 600000000 Units 4500 Tenants 0
2023-09-01T13:38:26.499+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9007 SG sg1 Memory 600000000 Units 4100 Tenants 0
2023-09-01T13:38:26.499+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 600000000 Units 4100 Tenants 0
2023-09-01T13:38:26.499+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9003 127.0.0.1:9006] [127.0.0.1:9007 127.0.0.1:9008]]
2023-09-01T13:38:26.499+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [[127.0.0.1:9002 127.0.0.1:9005]]
2023-09-01T13:38:26.499+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.499+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket10 MemoryUsage 200000000 UnitsUsage 800 
2023-09-01T13:38:26.499+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket7 MemoryUsage 25000000 UnitsUsage 500 
2023-09-01T13:38:26.499+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket9 MemoryUsage 100000000 UnitsUsage 700 
2023-09-01T13:38:26.499+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket15 MemoryUsage 0 UnitsUsage 0 
2023-09-01T13:38:26.499+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket8 MemoryUsage 75000000 UnitsUsage 1000 
2023-09-01T13:38:26.499+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket16 MemoryUsage 0 UnitsUsage 0 
2023-09-01T13:38:26.499+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters Below LWM []
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 4 SG, Move out 1 subcluster, Not Enough Capacity
2023-09-01T13:38:26.500+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.500+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.500+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9004 127.0.0.1:9001]
2023-09-01T13:38:26.500+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes []
2023-09-01T13:38:26.500+05:30 [Info] Planner::moveTenantsFromDeletedNodes Num deleted nodes 2 is more than num new/empty nodes 0
2023-09-01T13:38:26.500+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantsToBeMoved from source [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.500+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket5 MemoryUsage 300000000 UnitsUsage 2000 
2023-09-01T13:38:26.500+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket6 MemoryUsage 300000000 UnitsUsage 2000 
2023-09-01T13:38:26.500+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 120000000 UnitsUsage 1500 
2023-09-01T13:38:26.500+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket3 MemoryUsage 80000000 UnitsUsage 1200 
2023-09-01T13:38:26.500+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 70000000 UnitsUsage 800 
2023-09-01T13:38:26.500+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket1 MemoryUsage 30000000 UnitsUsage 500 
2023-09-01T13:38:26.500+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket17 MemoryUsage 0 UnitsUsage 0 
2023-09-01T13:38:26.500+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket14 MemoryUsage 0 UnitsUsage 0 
2023-09-01T13:38:26.500+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 8000
2023-09-01T13:38:26.500+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.500+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 100000000 Units 500 Tenants 0
2023-09-01T13:38:26.500+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 8000
2023-09-01T13:38:26.500+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.500+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 100000000 Units 500 Tenants 0
2023-09-01T13:38:26.500+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9007 SG sg1 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.500+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.500+05:30 [Info] Planner::moveTenantsFromDeletedNodes Found SubClusters Below LWM [[127.0.0.1:9003 127.0.0.1:9006] [127.0.0.1:9007 127.0.0.1:9008]]
2023-09-01T13:38:26.500+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket5 MemoryUsage 300000000 UnitsUsage 2000  can be placed on [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.500+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket6 MemoryUsage 300000000 UnitsUsage 2000  can be placed on [127.0.0.1:9007 127.0.0.1:9008]
2023-09-01T13:38:26.500+05:30 [Info] Planner::moveTenantsToLowUsageSubCluster Unable to place TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 120000000 UnitsUsage 1500  on any target
2023-09-01T13:38:26.500+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket3 MemoryUsage 80000000 UnitsUsage 1200  can be placed on [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.500+05:30 [Info] Planner::moveTenantsToLowUsageSubCluster Unable to place TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 70000000 UnitsUsage 800  on any target
2023-09-01T13:38:26.500+05:30 [Info] Planner::moveTenantsToLowUsageSubCluster Unable to place TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket1 MemoryUsage 30000000 UnitsUsage 500  on any target
2023-09-01T13:38:26.500+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket17 MemoryUsage 0 UnitsUsage 0  can be placed on [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.500+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket14 MemoryUsage 0 UnitsUsage 0  can be placed on [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.500+05:30 [Error] Planner - Not enough capacity to place indexes of deleted nodes.
2023/09/01 13:38:26 Expected error Planner - Not enough capacity to place indexes of deleted nodes.
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 4 SG, Move out 1 node, Pair node not deleted(Failed swap rebalance of 1 node)
2023-09-01T13:38:26.502+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9001]
2023-09-01T13:38:26.502+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9001]
2023-09-01T13:38:26.502+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9004]
2023-09-01T13:38:26.502+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes []
2023-09-01T13:38:26.502+05:30 [Info] Planner::moveTenantsFromDeletedNodes Num deleted nodes 1 is more than num new/empty nodes 0
2023-09-01T13:38:26.502+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 8000
2023-09-01T13:38:26.502+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.502+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 100000000 Units 500 Tenants 0
2023-09-01T13:38:26.502+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.502+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.502+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 100000000 Units 500 Tenants 0
2023-09-01T13:38:26.502+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9007 SG sg1 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.502+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.502+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9009 SG sg1 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.502+05:30 [Info] Planner::moveTenantsFromDeletedNodes Considering 127.0.0.1:9009 as replacement node for deleted node 127.0.0.1:9001.
2023-09-01T13:38:26.502+05:30 [Info] Moving index 1111:1212:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9009
2023-09-01T13:38:26.502+05:30 [Info] Moving index 2222:2121:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9009
2023-09-01T13:38:26.502+05:30 [Info] Moving index 3333:3131:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9009
2023-09-01T13:38:26.502+05:30 [Info] Moving index 4444:4141:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9009
2023-09-01T13:38:26.502+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9001 SG sg1 Memory 600000000 Units 4000
2023-09-01T13:38:26.502+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.502+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 100000000 Units 500 Tenants 0
2023-09-01T13:38:26.502+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.502+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.502+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 100000000 Units 500 Tenants 0
2023-09-01T13:38:26.502+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9007 SG sg1 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.502+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.502+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9009 SG sg1 Memory 500000000 Units 4100 Tenants 0
2023-09-01T13:38:26.502+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9003 127.0.0.1:9006] [127.0.0.1:9004 127.0.0.1:9009] [127.0.0.1:9007 127.0.0.1:9008]]
2023-09-01T13:38:26.502+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [[127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9004 127.0.0.1:9009]]
2023-09-01T13:38:26.502+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.502+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket10 MemoryUsage 200000000 UnitsUsage 800 
2023-09-01T13:38:26.502+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket7 MemoryUsage 25000000 UnitsUsage 500 
2023-09-01T13:38:26.502+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket9 MemoryUsage 100000000 UnitsUsage 700 
2023-09-01T13:38:26.503+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket15 MemoryUsage 0 UnitsUsage 0 
2023-09-01T13:38:26.503+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket8 MemoryUsage 75000000 UnitsUsage 1000 
2023-09-01T13:38:26.503+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket16 MemoryUsage 0 UnitsUsage 0 
2023-09-01T13:38:26.503+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9004 127.0.0.1:9009]
2023-09-01T13:38:26.503+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9004 TenantId bucket4 MemoryUsage 120000000 UnitsUsage 1500 
2023-09-01T13:38:26.503+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9004 TenantId bucket2 MemoryUsage 70000000 UnitsUsage 800 
2023-09-01T13:38:26.503+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9004 TenantId bucket3 MemoryUsage 80000000 UnitsUsage 1200 
2023-09-01T13:38:26.503+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9004 TenantId bucket1 MemoryUsage 30000000 UnitsUsage 500 
2023-09-01T13:38:26.503+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9004 TenantId bucket14 MemoryUsage 0 UnitsUsage 0 
2023-09-01T13:38:26.503+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9004 TenantId bucket17 MemoryUsage 0 UnitsUsage 0 
2023-09-01T13:38:26.503+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters Below LWM [[127.0.0.1:9003 127.0.0.1:9006] [127.0.0.1:9007 127.0.0.1:9008]]
2023-09-01T13:38:26.503+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket10 MemoryUsage 200000000 UnitsUsage 800  can be placed on [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.503+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9004 TenantId bucket4 MemoryUsage 120000000 UnitsUsage 1500  can be placed on [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.503+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket7 MemoryUsage 25000000 UnitsUsage 500  can be placed on [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.503+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9004 TenantId bucket2 MemoryUsage 70000000 UnitsUsage 800  can be placed on [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.503+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket9 MemoryUsage 100000000 UnitsUsage 700  can be placed on [127.0.0.1:9007 127.0.0.1:9008]
2023-09-01T13:38:26.503+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9004 TenantId bucket3 MemoryUsage 80000000 UnitsUsage 1200  can be placed on [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.503+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket15 MemoryUsage 0 UnitsUsage 0  can be placed on [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.503+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9004 TenantId bucket1 MemoryUsage 30000000 UnitsUsage 500  can be placed on [127.0.0.1:9007 127.0.0.1:9008]
2023-09-01T13:38:26.503+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket8 MemoryUsage 75000000 UnitsUsage 1000  can be placed on [127.0.0.1:9007 127.0.0.1:9008]
2023-09-01T13:38:26.503+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9004 TenantId bucket14 MemoryUsage 0 UnitsUsage 0  can be placed on [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.503+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket16 MemoryUsage 0 UnitsUsage 0  can be placed on [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.503+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9004 TenantId bucket17 MemoryUsage 0 UnitsUsage 0  can be placed on [127.0.0.1:9003 127.0.0.1:9006]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 4 SG, Move out 1 node, Pair node already deleted
2023-09-01T13:38:26.504+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9002]
2023-09-01T13:38:26.504+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9002]
2023-09-01T13:38:26.504+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes []
2023-09-01T13:38:26.504+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes []
2023-09-01T13:38:26.504+05:30 [Info] Planner::moveTenantsFromDeletedNodes Num deleted nodes 1 is more than num new/empty nodes 0
2023-09-01T13:38:26.504+05:30 [Info] Planner::moveTenantsFromDeletedNodes Pair node not found for deleted node 127.0.0.1:9002.
2023-09-01T13:38:26.504+05:30 [Error] Planner - Pair node for 127.0.0.1:9002 not found. Provide additional node as replacement.
2023/09/01 13:38:26 Expected error Planner - Pair node for 127.0.0.1:9002 not found. Provide additional node as replacement.
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 4 SG, Move out 1 subcluster, empty nodes
2023-09-01T13:38:26.505+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9005 127.0.0.1:9007 127.0.0.1:9008]
2023-09-01T13:38:26.505+05:30 [Info] Planner::moveTenantsFromDeletedNodes No non-empty deleted nodes found.
2023-09-01T13:38:26.505+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9005 SG sg3 Memory 0 Units 0
2023-09-01T13:38:26.505+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9007 SG sg1 Memory 200000000 Units 100
2023-09-01T13:38:26.505+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9008 SG sg2 Memory 200000000 Units 100
2023-09-01T13:38:26.505+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 800000000 Units 8000 Tenants 0
2023-09-01T13:38:26.505+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 800000000 Units 7000 Tenants 0
2023-09-01T13:38:26.505+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 100000000 Units 500 Tenants 0
2023-09-01T13:38:26.505+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 800000000 Units 8000 Tenants 0
2023-09-01T13:38:26.505+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 100000000 Units 500 Tenants 0
2023-09-01T13:38:26.505+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002] [127.0.0.1:9003 127.0.0.1:9006]]
2023-09-01T13:38:26.505+05:30 [Info] Planner::repairMissingReplica Found SubCluster [127.0.0.1:9002] with len 1. Skipping replica repair attempt.
2023-09-01T13:38:26.505+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM []
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 4 SG, Move out 1 subcluster, Deleted Nodes more than added nodes
2023-09-01T13:38:26.506+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.506+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.506+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9004 127.0.0.1:9001]
2023-09-01T13:38:26.506+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes [127.0.0.1:9008]
2023-09-01T13:38:26.506+05:30 [Info] Planner::moveTenantsFromDeletedNodes Num deleted nodes 2 is more than num new/empty nodes 1
2023-09-01T13:38:26.506+05:30 [Error] Planner - Number of non-empty deleted nodes cannot be greater than number of added nodes.
2023/09/01 13:38:26 Expected error Planner - Number of non-empty deleted nodes cannot be greater than number of added nodes.
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 4 SG, Move out 1 node, Add 1 node in, server group mismatch
2023-09-01T13:38:26.507+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9006]
2023-09-01T13:38:26.507+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9006]
2023-09-01T13:38:26.507+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9003]
2023-09-01T13:38:26.507+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes [127.0.0.1:9007]
2023/09/01 13:38:26 Expected error Planner - Unable to satisfy server group constraint while replacing removed nodes with new nodes.
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 4 SG, Move out 1 node, Pair node exists
2023-09-01T13:38:26.507+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9004]
2023-09-01T13:38:26.507+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9004]
2023-09-01T13:38:26.507+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9001]
2023-09-01T13:38:26.507+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes []
2023-09-01T13:38:26.507+05:30 [Info] Planner::moveTenantsFromDeletedNodes Num deleted nodes 1 is more than num new/empty nodes 0
2023-09-01T13:38:26.507+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.507+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.507+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 100000000 Units 500 Tenants 0
2023-09-01T13:38:26.507+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 8000
2023-09-01T13:38:26.507+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.508+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 100000000 Units 500 Tenants 0
2023-09-01T13:38:26.508+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9007 SG sg1 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.508+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.508+05:30 [Info] Planner::moveTenantsFromDeletedNodes No replacement node found for deleted node 127.0.0.1:9004.
2023-09-01T13:38:26.508+05:30 [Error] Planner - Removing node 127.0.0.1:9004 will result in losing indexes. Provide additional node as replacement.
2023/09/01 13:38:26 Expected error Planner - Removing node 127.0.0.1:9004 will result in losing indexes. Provide additional node as replacement.
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 4 SG, Move out 1 subcluster, No nodes under LWM
2023-09-01T13:38:26.508+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.508+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.508+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9004 127.0.0.1:9001]
2023-09-01T13:38:26.508+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes []
2023-09-01T13:38:26.508+05:30 [Info] Planner::moveTenantsFromDeletedNodes Num deleted nodes 2 is more than num new/empty nodes 0
2023-09-01T13:38:26.508+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantsToBeMoved from source [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.508+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket5 MemoryUsage 300000000 UnitsUsage 2000 
2023-09-01T13:38:26.508+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket6 MemoryUsage 300000000 UnitsUsage 2000 
2023-09-01T13:38:26.508+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 120000000 UnitsUsage 1500 
2023-09-01T13:38:26.508+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket3 MemoryUsage 80000000 UnitsUsage 1200 
2023-09-01T13:38:26.508+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 70000000 UnitsUsage 800 
2023-09-01T13:38:26.508+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket1 MemoryUsage 30000000 UnitsUsage 500 
2023-09-01T13:38:26.508+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket17 MemoryUsage 0 UnitsUsage 0 
2023-09-01T13:38:26.508+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket14 MemoryUsage 0 UnitsUsage 0 
2023-09-01T13:38:26.508+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 8000
2023-09-01T13:38:26.508+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.508+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 8000
2023-09-01T13:38:26.509+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.509+05:30 [Info] Planner::moveTenantsFromDeletedNodes Found SubClusters Below LWM []
2023-09-01T13:38:26.509+05:30 [Error] Planner - Not enough capacity to place indexes of deleted nodes.
2023/09/01 13:38:26 Expected error Planner - Not enough capacity to place indexes of deleted nodes.
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 4 SG, Move out 1 subcluster, Not Enough Capacity, Partial Subcluster
2023-09-01T13:38:26.509+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.509+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.509+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9004 127.0.0.1:9001]
2023-09-01T13:38:26.509+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes []
2023-09-01T13:38:26.509+05:30 [Info] Planner::moveTenantsFromDeletedNodes Num deleted nodes 2 is more than num new/empty nodes 0
2023-09-01T13:38:26.509+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantsToBeMoved from source [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.509+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket1 MemoryUsage 100000000 UnitsUsage 500 
2023-09-01T13:38:26.509+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 100000000 UnitsUsage 500 
2023-09-01T13:38:26.509+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9001 SG sg1 Memory 200000000 Units 1000
2023-09-01T13:38:26.509+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.509+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9004 SG sg2 Memory 200000000 Units 1000
2023-09-01T13:38:26.509+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.509+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 100000000 Units 500 Tenants 0
2023-09-01T13:38:26.509+05:30 [Info] Planner::filterPartialSubClusters Filter partial subcluster [127.0.0.1:9006]
2023-09-01T13:38:26.509+05:30 [Info] Planner::moveTenantsFromDeletedNodes Found SubClusters Below LWM []
2023-09-01T13:38:26.509+05:30 [Error] Planner - Not enough capacity to place indexes of deleted nodes.
2023/09/01 13:38:26 Expected error Planner - Not enough capacity to place indexes of deleted nodes.
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 2 SG, Move out 1 non-empty and 1 empty  node
2023-09-01T13:38:26.510+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9007 127.0.0.1:9004]
2023-09-01T13:38:26.510+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9004]
2023-09-01T13:38:26.510+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9001]
2023-09-01T13:38:26.510+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes [127.0.0.1:9008]
2023-09-01T13:38:26.510+05:30 [Info] Planner::moveTenantsFromDeletedNodes selected newNodes for swap [127.0.0.1:9008]
2023-09-01T13:38:26.510+05:30 [Info] Moving index 1111:11212:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9008
2023-09-01T13:38:26.510+05:30 [Info] Moving index 2222:22121:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9008
2023-09-01T13:38:26.510+05:30 [Info] Moving index 3333:33131:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9008
2023-09-01T13:38:26.510+05:30 [Info] Moving index 4444:44141:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9008
2023-09-01T13:38:26.510+05:30 [Info] Moving index 5555:55151:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9008
2023-09-01T13:38:26.510+05:30 [Info] Moving index 6666:66161:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9008
2023-09-01T13:38:26.510+05:30 [Info] Moving index 171717:1171171:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9008
2023-09-01T13:38:26.510+05:30 [Info] Moving index 141414:1141141:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9008
2023-09-01T13:38:26.510+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9007 SG sg1 Memory 0 Units 0
2023-09-01T13:38:26.510+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9004 SG sg2 Memory 0 Units 0
2023-09-01T13:38:26.510+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 800000000 Units 8000 Tenants 0
2023-09-01T13:38:26.510+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 800000000 Units 8000 Tenants 0
2023-09-01T13:38:26.511+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9008 127.0.0.1:9001]]
2023-09-01T13:38:26.511+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM []
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 2 Subclusters, 1 empty, 1 Above HWM
2023-09-01T13:38:26.511+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 9000 Tenants 0
2023-09-01T13:38:26.511+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 9000 Tenants 0
2023-09-01T13:38:26.511+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.512+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [[127.0.0.1:9001 127.0.0.1:9004]]
2023-09-01T13:38:26.512+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.512+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 300000000 UnitsUsage 3000 
2023-09-01T13:38:26.512+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 200000000 UnitsUsage 2000 
2023-09-01T13:38:26.512+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters Below LWM [[127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.512+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 300000000 UnitsUsage 3000  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.512+05:30 [Info] Planner::moveTenantsToLowUsageSubCluster Unable to place TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 200000000 UnitsUsage 2000  on any target
2023-09-01T13:38:26.512+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 600000000 Units 6000 Tenants 0
2023-09-01T13:38:26.512+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 300000000 Units 3000 Tenants 0
2023-09-01T13:38:26.512+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 600000000 Units 6000 Tenants 0
2023-09-01T13:38:26.512+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 300000000 Units 3000 Tenants 0
2023-09-01T13:38:26.512+05:30 [Info] Planner::evaluateSolutionForScaleIn Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9003 127.0.0.1:9006]]
2023-09-01T13:38:26.512+05:30 [Info] Planner::evaluateSolutionForScaleIn Found SubClusters below LWM [[127.0.0.1:9003 127.0.0.1:9006]]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 2 Subclusters, 1 below LWM, 1 above HWM
2023-09-01T13:38:26.513+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 9000 Tenants 0
2023-09-01T13:38:26.513+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.513+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 9000 Tenants 0
2023-09-01T13:38:26.513+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.513+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005]]
2023-09-01T13:38:26.513+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [[127.0.0.1:9001 127.0.0.1:9004]]
2023-09-01T13:38:26.513+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.513+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 300000000 UnitsUsage 3000 
2023-09-01T13:38:26.513+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 200000000 UnitsUsage 2000 
2023-09-01T13:38:26.513+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters Below LWM [[127.0.0.1:9002 127.0.0.1:9005]]
2023-09-01T13:38:26.513+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 300000000 UnitsUsage 3000  can be placed on [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.513+05:30 [Info] Planner::moveTenantsToLowUsageSubCluster Unable to place TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 200000000 UnitsUsage 2000  on any target
2023-09-01T13:38:26.513+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 600000000 Units 6000 Tenants 0
2023-09-01T13:38:26.513+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 600000000 Units 4000 Tenants 0
2023-09-01T13:38:26.513+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 600000000 Units 6000 Tenants 0
2023-09-01T13:38:26.513+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 600000000 Units 4000 Tenants 0
2023-09-01T13:38:26.513+05:30 [Info] Planner::evaluateSolutionForScaleIn Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005]]
2023-09-01T13:38:26.513+05:30 [Info] Planner::evaluateSolutionForScaleIn Found SubClusters below LWM []
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 3 Subclusters, 1 empty, 1 Above HWM, 1 below LWM
2023-09-01T13:38:26.514+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 9000 Tenants 0
2023-09-01T13:38:26.514+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.514+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 9000 Tenants 0
2023-09-01T13:38:26.514+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 300000000 Units 1000 Tenants 0
2023-09-01T13:38:26.514+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.514+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [[127.0.0.1:9001 127.0.0.1:9004]]
2023-09-01T13:38:26.514+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.514+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 300000000 UnitsUsage 3000 
2023-09-01T13:38:26.514+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 200000000 UnitsUsage 2000 
2023-09-01T13:38:26.514+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters Below LWM [[127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.514+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 300000000 UnitsUsage 3000  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.514+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 200000000 UnitsUsage 2000  can be placed on [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.514+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 400000000 Units 4000 Tenants 0
2023-09-01T13:38:26.514+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 500000000 Units 3000 Tenants 0
2023-09-01T13:38:26.514+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 300000000 Units 3000 Tenants 0
2023-09-01T13:38:26.514+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 400000000 Units 4000 Tenants 0
2023-09-01T13:38:26.514+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 500000000 Units 3000 Tenants 0
2023-09-01T13:38:26.514+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 300000000 Units 3000 Tenants 0
2023-09-01T13:38:26.514+05:30 [Info] Planner::evaluateSolutionForScaleIn Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9003 127.0.0.1:9006]]
2023-09-01T13:38:26.514+05:30 [Info] Planner::evaluateSolutionForScaleIn Found SubClusters below LWM [[127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9003 127.0.0.1:9006]]
2023-09-01T13:38:26.514+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.514+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.514+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.514+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes []
2023-09-01T13:38:26.514+05:30 [Info] Planner::moveTenantsFromDeletedNodes Num deleted nodes 2 is more than num new/empty nodes 0
2023-09-01T13:38:26.514+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantsToBeMoved from source [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.514+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9003 TenantId bucket4 MemoryUsage 300000000 UnitsUsage 3000 
2023-09-01T13:38:26.514+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 400000000 Units 4000 Tenants 0
2023-09-01T13:38:26.514+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 500000000 Units 3000 Tenants 0
2023-09-01T13:38:26.514+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9003 SG sg1 Memory 300000000 Units 3000
2023-09-01T13:38:26.515+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 400000000 Units 4000 Tenants 0
2023-09-01T13:38:26.515+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 500000000 Units 3000 Tenants 0
2023-09-01T13:38:26.515+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9006 SG sg3 Memory 300000000 Units 3000
2023-09-01T13:38:26.515+05:30 [Info] Planner::moveTenantsFromDeletedNodes Found SubClusters Below LWM []
2023-09-01T13:38:26.515+05:30 [Error] Planner - Not enough capacity to place indexes of deleted nodes.
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 2 Subclusters, 1 above LWM/below HWM, 1 empty
2023-09-01T13:38:26.516+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 700000000 Units 5000 Tenants 0
2023-09-01T13:38:26.516+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 700000000 Units 5000 Tenants 0
2023-09-01T13:38:26.516+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.516+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM []
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 2 Subclusters, Both above LWM/below HWM
2023-09-01T13:38:26.516+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 700000000 Units 5000 Tenants 0
2023-09-01T13:38:26.517+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 650000000 Units 4500 Tenants 0
2023-09-01T13:38:26.517+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 700000000 Units 5000 Tenants 0
2023-09-01T13:38:26.517+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 650000000 Units 4500 Tenants 0
2023-09-01T13:38:26.517+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9003 127.0.0.1:9006]]
2023-09-01T13:38:26.517+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM []
2023-09-01T13:38:26.517+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 700000000 Units 5000 Tenants 0
2023-09-01T13:38:26.517+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 650000000 Units 4500 Tenants 0
2023-09-01T13:38:26.517+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 700000000 Units 5000 Tenants 0
2023-09-01T13:38:26.517+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 650000000 Units 4500 Tenants 0
2023-09-01T13:38:26.517+05:30 [Info] Planner::evaluateSolutionForScaleIn Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9003 127.0.0.1:9006]]
2023-09-01T13:38:26.517+05:30 [Info] Planner::evaluateSolutionForScaleIn Found SubClusters below LWM []
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 2 Subclusters, 1 empty, 1 Above HWM (partial replica repair)
2023-09-01T13:38:26.517+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 9000 Tenants 0
2023-09-01T13:38:26.517+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 550000000 Units 5500 Tenants 0
2023-09-01T13:38:26.517+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.517+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket4,,,idx2,0) on 127.0.0.1:9004
2023-09-01T13:38:26.517+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket5,,,idx2,0) on 127.0.0.1:9004
2023-09-01T13:38:26.517+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [[127.0.0.1:9001 127.0.0.1:9004]]
2023-09-01T13:38:26.517+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.517+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 300000000 UnitsUsage 3000 
2023-09-01T13:38:26.517+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 200000000 UnitsUsage 2000 
2023-09-01T13:38:26.518+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters Below LWM [[127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.518+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 300000000 UnitsUsage 3000  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.518+05:30 [Info] Planner::moveTenantsToLowUsageSubCluster Unable to place TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 200000000 UnitsUsage 2000  on any target
2023-09-01T13:38:26.518+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 600000000 Units 6000 Tenants 0
2023-09-01T13:38:26.518+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 300000000 Units 3000 Tenants 0
2023-09-01T13:38:26.518+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 600000000 Units 6000 Tenants 0
2023-09-01T13:38:26.518+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 300000000 Units 3000 Tenants 0
2023-09-01T13:38:26.518+05:30 [Info] Planner::evaluateSolutionForScaleIn Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9003 127.0.0.1:9006]]
2023-09-01T13:38:26.518+05:30 [Info] Planner::evaluateSolutionForScaleIn Found SubClusters below LWM [[127.0.0.1:9003 127.0.0.1:9006]]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 2 Subclusters, 1 empty, 1 Above HWM(full replica repair)
2023-09-01T13:38:26.518+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 9000 Tenants 0
2023-09-01T13:38:26.518+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9004 127.0.0.1:9001] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.518+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket2,,,idx1,1) on 127.0.0.1:9001
2023-09-01T13:38:26.518+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket2,,,idx2,1) on 127.0.0.1:9001
2023-09-01T13:38:26.518+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket4,,,idx1,1) on 127.0.0.1:9001
2023-09-01T13:38:26.518+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket4,,,idx2,1) on 127.0.0.1:9001
2023-09-01T13:38:26.518+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket5,,,idx1,1) on 127.0.0.1:9001
2023-09-01T13:38:26.518+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket5,,,idx2,1) on 127.0.0.1:9001
2023-09-01T13:38:26.518+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [[127.0.0.1:9004 127.0.0.1:9001]]
2023-09-01T13:38:26.518+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9004 127.0.0.1:9001]
2023-09-01T13:38:26.518+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9004 TenantId bucket4 MemoryUsage 300000000 UnitsUsage 3000 
2023-09-01T13:38:26.518+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9004 TenantId bucket2 MemoryUsage 200000000 UnitsUsage 2000 
2023-09-01T13:38:26.518+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters Below LWM [[127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.518+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9004 TenantId bucket4 MemoryUsage 300000000 UnitsUsage 3000  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.518+05:30 [Info] Planner::moveTenantsToLowUsageSubCluster Unable to place TenantUsage - SourceId 127.0.0.1:9004 TenantId bucket2 MemoryUsage 200000000 UnitsUsage 2000  on any target
2023-09-01T13:38:26.518+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 600000000 Units 6000 Tenants 0
2023-09-01T13:38:26.518+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 300000000 Units 3000 Tenants 0
2023-09-01T13:38:26.518+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 600000000 Units 6000 Tenants 0
2023-09-01T13:38:26.519+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 300000000 Units 3000 Tenants 0
2023-09-01T13:38:26.519+05:30 [Info] Planner::evaluateSolutionForScaleIn Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9003 127.0.0.1:9006]]
2023-09-01T13:38:26.519+05:30 [Info] Planner::evaluateSolutionForScaleIn Found SubClusters below LWM [[127.0.0.1:9003 127.0.0.1:9006]]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 ScaleIn- 2 Subclusters, Both below LWM, Positive Case
2023-09-01T13:38:26.519+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 200000000 Units 1000 Tenants 0
2023-09-01T13:38:26.519+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 100000000 Units 2000 Tenants 0
2023-09-01T13:38:26.519+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 200000000 Units 1000 Tenants 0
2023-09-01T13:38:26.519+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 100000000 Units 2000 Tenants 0
2023-09-01T13:38:26.519+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9003 127.0.0.1:9006]]
2023-09-01T13:38:26.519+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM []
2023-09-01T13:38:26.519+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 200000000 Units 1000 Tenants 0
2023-09-01T13:38:26.519+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 100000000 Units 2000 Tenants 0
2023-09-01T13:38:26.519+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 200000000 Units 1000 Tenants 0
2023-09-01T13:38:26.519+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 100000000 Units 2000 Tenants 0
2023-09-01T13:38:26.519+05:30 [Info] Planner::evaluateSolutionForScaleIn Found SubClusters  [[127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9003 127.0.0.1:9006]]
2023-09-01T13:38:26.519+05:30 [Info] Planner::evaluateSolutionForScaleIn Found SubClusters below LWM [[127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9003 127.0.0.1:9006]]
2023-09-01T13:38:26.519+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.519+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.519+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.519+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes []
2023-09-01T13:38:26.519+05:30 [Info] Planner::moveTenantsFromDeletedNodes Num deleted nodes 2 is more than num new/empty nodes 0
2023-09-01T13:38:26.519+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantsToBeMoved from source [127.0.0.1:9003 127.0.0.1:9006]
2023-09-01T13:38:26.519+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9003 TenantId bucket3 MemoryUsage 100000000 UnitsUsage 2000 
2023-09-01T13:38:26.519+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 200000000 Units 1000 Tenants 0
2023-09-01T13:38:26.519+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9003 SG sg1 Memory 100000000 Units 2000
2023-09-01T13:38:26.519+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 200000000 Units 1000 Tenants 0
2023-09-01T13:38:26.519+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9006 SG sg3 Memory 100000000 Units 2000
2023-09-01T13:38:26.519+05:30 [Info] Planner::moveTenantsFromDeletedNodes Found SubClusters Below LWM [[127.0.0.1:9002 127.0.0.1:9005]]
2023-09-01T13:38:26.519+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9003 TenantId bucket3 MemoryUsage 100000000 UnitsUsage 2000  can be placed on [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.519+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9003 SG sg1 Memory 0 Units 0
2023-09-01T13:38:26.519+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9006 SG sg3 Memory 0 Units 0
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 ScaleIn- 2 Subclusters, One below LWM/ 1 Empty
2023-09-01T13:38:26.520+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 200000000 Units 1000 Tenants 0
2023-09-01T13:38:26.520+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 200000000 Units 1000 Tenants 0
2023-09-01T13:38:26.520+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.520+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM []
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 ScaleIn- 3 Subclusters, One above HWM, one below LWM and 1 Empty. No ScaleIn.
2023-09-01T13:38:26.521+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 800000000 Units 8000 Tenants 0
2023-09-01T13:38:26.521+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.521+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 800000000 Units 8000 Tenants 0
2023-09-01T13:38:26.521+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.521+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.521+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [[127.0.0.1:9001 127.0.0.1:9004]]
2023-09-01T13:38:26.521+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.521+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 300000000 UnitsUsage 3000 
2023-09-01T13:38:26.521+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 100000000 UnitsUsage 1000 
2023-09-01T13:38:26.521+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters Below LWM [[127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.521+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 300000000 UnitsUsage 3000  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.521+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 100000000 UnitsUsage 1000  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.521+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 400000000 Units 4000 Tenants 0
2023-09-01T13:38:26.521+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.521+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 400000000 Units 4000 Tenants 0
2023-09-01T13:38:26.521+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 400000000 Units 4000 Tenants 0
2023-09-01T13:38:26.521+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.521+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 400000000 Units 4000 Tenants 0
2023-09-01T13:38:26.521+05:30 [Info] Planner::evaluateSolutionForScaleIn Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9003 127.0.0.1:9006]]
2023-09-01T13:38:26.521+05:30 [Info] Planner::evaluateSolutionForScaleIn Found SubClusters below LWM [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9003 127.0.0.1:9006]]
2023-09-01T13:38:26.521+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.521+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.521+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9005 127.0.0.1:9002]
2023-09-01T13:38:26.521+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes []
2023-09-01T13:38:26.521+05:30 [Info] Planner::moveTenantsFromDeletedNodes Num deleted nodes 2 is more than num new/empty nodes 0
2023-09-01T13:38:26.521+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantsToBeMoved from source [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.521+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket3 MemoryUsage 100000000 UnitsUsage 1000 
2023-09-01T13:38:26.521+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 400000000 Units 4000 Tenants 0
2023-09-01T13:38:26.521+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9002 SG sg2 Memory 100000000 Units 1000
2023-09-01T13:38:26.521+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 400000000 Units 4000 Tenants 0
2023-09-01T13:38:26.521+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 400000000 Units 4000 Tenants 0
2023-09-01T13:38:26.521+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9005 SG sg3 Memory 100000000 Units 1000
2023-09-01T13:38:26.521+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 400000000 Units 4000 Tenants 0
2023-09-01T13:38:26.521+05:30 [Info] Planner::moveTenantsFromDeletedNodes Found SubClusters Below LWM []
2023-09-01T13:38:26.521+05:30 [Error] Planner - Not enough capacity to place indexes of deleted nodes.
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 ScaleIn- 3 Subclusters, One above HWM, one below LWM and 1 Empty. ScaleIn. 
2023-09-01T13:38:26.522+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 800000000 Units 8000 Tenants 0
2023-09-01T13:38:26.522+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.522+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 800000000 Units 8000 Tenants 0
2023-09-01T13:38:26.522+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.522+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.522+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [[127.0.0.1:9001 127.0.0.1:9004]]
2023-09-01T13:38:26.522+05:30 [Info] Planner::executeTenantAwareRebal TenantsToBeMoved from source [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.522+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 300000000 UnitsUsage 3000 
2023-09-01T13:38:26.522+05:30 [Info] Planner::executeTenantAwareRebal TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 100000000 UnitsUsage 1000 
2023-09-01T13:38:26.522+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters Below LWM [[127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.522+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket4 MemoryUsage 300000000 UnitsUsage 3000  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.522+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9001 TenantId bucket2 MemoryUsage 100000000 UnitsUsage 1000  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023-09-01T13:38:26.522+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 400000000 Units 4000 Tenants 0
2023-09-01T13:38:26.522+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.522+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 400000000 Units 4000 Tenants 0
2023-09-01T13:38:26.522+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 400000000 Units 4000 Tenants 0
2023-09-01T13:38:26.522+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 100000000 Units 1000 Tenants 0
2023-09-01T13:38:26.522+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 400000000 Units 4000 Tenants 0
2023-09-01T13:38:26.522+05:30 [Info] Planner::evaluateSolutionForScaleIn Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9003 127.0.0.1:9006]]
2023-09-01T13:38:26.522+05:30 [Info] Planner::evaluateSolutionForScaleIn Found SubClusters below LWM [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9003 127.0.0.1:9006]]
2023-09-01T13:38:26.522+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.522+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.522+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9005 127.0.0.1:9002]
2023-09-01T13:38:26.522+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes []
2023-09-01T13:38:26.522+05:30 [Info] Planner::moveTenantsFromDeletedNodes Num deleted nodes 2 is more than num new/empty nodes 0
2023-09-01T13:38:26.522+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantsToBeMoved from source [127.0.0.1:9002 127.0.0.1:9005]
2023-09-01T13:38:26.522+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket3 MemoryUsage 100000000 UnitsUsage 1000 
2023-09-01T13:38:26.522+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 400000000 Units 4000 Tenants 0
2023-09-01T13:38:26.522+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9002 SG sg2 Memory 100000000 Units 1000
2023-09-01T13:38:26.522+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG sg1 Memory 400000000 Units 4000 Tenants 0
2023-09-01T13:38:26.522+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 400000000 Units 4000 Tenants 0
2023-09-01T13:38:26.522+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9005 SG sg3 Memory 100000000 Units 1000
2023-09-01T13:38:26.522+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9006 SG sg3 Memory 400000000 Units 4000 Tenants 0
2023-09-01T13:38:26.522+05:30 [Info] Planner::moveTenantsFromDeletedNodes Found SubClusters Below LWM [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9003 127.0.0.1:9006]]
2023-09-01T13:38:26.523+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket3 MemoryUsage 100000000 UnitsUsage 1000  can be placed on [127.0.0.1:9001 127.0.0.1:9004]
2023-09-01T13:38:26.523+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9002 SG sg2 Memory 0 Units 0
2023-09-01T13:38:26.523+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9005 SG sg3 Memory 0 Units 0
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Rebalance - 1 Subcluster, Below HWM (partial replica repair)
2023-09-01T13:38:26.523+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 500000000 Units 800 Tenants 0
2023-09-01T13:38:26.523+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 350000000 Units 500 Tenants 0
2023-09-01T13:38:26.523+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004]]
2023-09-01T13:38:26.523+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket4,,,idx2,0) on 127.0.0.1:9004
2023-09-01T13:38:26.523+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket5,,,idx2,0) on 127.0.0.1:9004
2023-09-01T13:38:26.523+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM []
2023-09-01T13:38:26.523+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 500000000 Units 800 Tenants 0
2023-09-01T13:38:26.523+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 500000000 Units 800 Tenants 0
2023-09-01T13:38:26.523+05:30 [Info] Planner::evaluateSolutionForScaleIn Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004]]
2023-09-01T13:38:26.523+05:30 [Info] Planner::evaluateSolutionForScaleIn Found SubClusters below LWM [[127.0.0.1:9001 127.0.0.1:9004]]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Resume - 1 tenant, Empty node in cluster.
2023-09-01T13:38:26.525+05:30 [Info] Planner::executeTenantAwarePlanForResume Resume Nodes [127.0.0.1:9011 127.0.0.1:9012]
2023-09-01T13:38:26.525+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.525+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.525+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.525+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.525+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9007 SG sg1 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.525+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.525+05:30 [Info] Planner::executeTenantAwarePlanForResume Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9007 127.0.0.1:9008] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.525+05:30 [Info] Planner::executeTenantAwarePlanForResume TenantToBeResumed TenantUsage - SourceId 127.0.0.1:9011 TenantId resume_tenant_a MemoryUsage 250000000 UnitsUsage 250 
2023-09-01T13:38:26.525+05:30 [Info] Planner::executeTenantAwarePlanForResume Found SubClusters Below LWM [[127.0.0.1:9007 127.0.0.1:9008] [127.0.0.1:9006 127.0.0.1:9003]]
2023-09-01T13:38:26.525+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9011 TenantId resume_tenant_a MemoryUsage 250000000 UnitsUsage 250  can be placed on [127.0.0.1:9006 127.0.0.1:9003]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Resume - 1 tenant, No empty node in cluster.
2023-09-01T13:38:26.527+05:30 [Info] Planner::executeTenantAwarePlanForResume Resume Nodes [127.0.0.1:9011 127.0.0.1:9012]
2023-09-01T13:38:26.527+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.527+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.527+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.527+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.527+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9007 SG sg1 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.527+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.527+05:30 [Info] Planner::executeTenantAwarePlanForResume Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9007 127.0.0.1:9008]]
2023-09-01T13:38:26.527+05:30 [Info] Planner::executeTenantAwarePlanForResume TenantToBeResumed TenantUsage - SourceId 127.0.0.1:9011 TenantId resume_tenant_a MemoryUsage 250000000 UnitsUsage 250 
2023-09-01T13:38:26.527+05:30 [Info] Planner::executeTenantAwarePlanForResume Found SubClusters Below LWM [[127.0.0.1:9007 127.0.0.1:9008]]
2023-09-01T13:38:26.527+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9011 TenantId resume_tenant_a MemoryUsage 250000000 UnitsUsage 250  can be placed on [127.0.0.1:9007 127.0.0.1:9008]
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Resume - 1 tenant, No node below LWM in the cluster.
2023-09-01T13:38:26.529+05:30 [Info] Planner::executeTenantAwarePlanForResume Resume Nodes [127.0.0.1:9011 127.0.0.1:9012]
2023-09-01T13:38:26.529+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.529+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.529+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.529+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.529+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9007 SG sg1 Memory 200000000 Units 5000 Tenants 0
2023-09-01T13:38:26.529+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 200000000 Units 5000 Tenants 0
2023-09-01T13:38:26.529+05:30 [Info] Planner::executeTenantAwarePlanForResume Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9007 127.0.0.1:9008]]
2023-09-01T13:38:26.529+05:30 [Info] Planner::executeTenantAwarePlanForResume TenantToBeResumed TenantUsage - SourceId 127.0.0.1:9011 TenantId resume_tenant_a MemoryUsage 250000000 UnitsUsage 250 
2023-09-01T13:38:26.529+05:30 [Info] Planner::executeTenantAwarePlanForResume Found SubClusters Below LWM []
2023/09/01 13:38:26 Expected error No SubCluster Below Low Usage Threshold
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Resume - 1 tenant, Not enough capacity in the cluster.
2023-09-01T13:38:26.530+05:30 [Info] Planner::executeTenantAwarePlanForResume Resume Nodes [127.0.0.1:9011 127.0.0.1:9012]
2023-09-01T13:38:26.530+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9001 SG sg1 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.530+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG sg2 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.530+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG sg2 Memory 900000000 Units 8000 Tenants 0
2023-09-01T13:38:26.530+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9005 SG sg3 Memory 1000000000 Units 7000 Tenants 0
2023-09-01T13:38:26.530+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9007 SG sg1 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.530+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9008 SG sg2 Memory 200000000 Units 100 Tenants 0
2023-09-01T13:38:26.530+05:30 [Info] Planner::executeTenantAwarePlanForResume Found SubClusters  [[127.0.0.1:9001 127.0.0.1:9004] [127.0.0.1:9002 127.0.0.1:9005] [127.0.0.1:9007 127.0.0.1:9008]]
2023-09-01T13:38:26.530+05:30 [Info] Planner::executeTenantAwarePlanForResume TenantToBeResumed TenantUsage - SourceId 127.0.0.1:9011 TenantId resume_tenant_a MemoryUsage 700000000 UnitsUsage 250 
2023-09-01T13:38:26.530+05:30 [Info] Planner::executeTenantAwarePlanForResume Found SubClusters Below LWM [[127.0.0.1:9007 127.0.0.1:9008]]
2023-09-01T13:38:26.530+05:30 [Info] Planner::moveTenantsToLowUsageSubCluster Unable to place TenantUsage - SourceId 127.0.0.1:9011 TenantId resume_tenant_a MemoryUsage 700000000 UnitsUsage 250  on any target
2023/09/01 13:38:26 Expected error Not Enough Capacity To Place Tenant
--- PASS: TestTenantAwarePlanner (0.11s)
=== RUN   TestPlanDuringHeterogenousScaleup
2023/09/01 13:38:26 In TestPlanDuringHeterogenousScaleup()
=== RUN   TestPlanDuringHeterogenousScaleup/GreedyPlanner
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 3 non-empty heterogenous nodes with 1 Scaled up - 1 SG
2023-09-01T13:38:26.533+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Index with 1 replica - 3 non-empty heterogenous nodes with 1 Scaled up - 1 SG
2023-09-01T13:38:26.534+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Index with 2 replicas - 3 non-empty heterogenous nodes with 1 Scaled up - 1 SG
2023-09-01T13:38:26.537+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Index With 1 Replica - 3 non-empty heterogenous nodes with 1 Scaled up - 2 SG - Placed on Scaled up node
2023-09-01T13:38:26.539+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Index With 1 Replica - 3 non-empty heterogenous nodes with 1 Scaled up - 2 SG - Placed on older nodes
2023-09-01T13:38:26.541+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Equivalent Index Without any replica - 3 non-empty heterogenous nodes with 1 Scaled up - 1 SG
2023-09-01T13:38:26.543+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Equivalent Index With 1 Replica - 3 non-empty heterogenous nodes with 1 Scaled up - 1 SG - Use least loaded node
2023-09-01T13:38:26.545+05:30 [Info] Using greedy index placement for index 987654
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Equivalent Index With 1 Replica - 3 non-empty heterogenous nodes with 1 Scaled up - 1 SG - Use ScaleUp Node 
2023-09-01T13:38:26.547+05:30 [Info] Using greedy index placement for index 987654
=== RUN   TestPlanDuringHeterogenousScaleup/SAPlanner
=== RUN   TestPlanDuringHeterogenousScaleup/SAPlanner/FunctionalDDLTests
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Index Instance - 3 non-empty heterogenous nodes with 1 Scaled up - 1 SG
2023-09-01T13:38:26.549+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 1.2180081556498616
2023-09-01T13:38:26.652+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:26.652+05:30 [Info] Score: 0.5118702120133598
2023-09-01T13:38:26.652+05:30 [Info] variation: 1.0237404240267196
2023-09-01T13:38:26.652+05:30 [Info] ElapsedTime: 102ms
2023-09-01T13:38:26.652+05:30 [Info] ConvergenceTime: 100ms
2023-09-01T13:38:26.652+05:30 [Info] Iteration: 7000
2023-09-01T13:38:26.652+05:30 [Info] Move: 3877
2023-09-01T13:38:26.652+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.652+05:30 [Info] Using SAPlanner
2023-09-01T13:38:26.652+05:30 [Info] Indexer Memory Mean 126888575449 (118.174G)
2023-09-01T13:38:26.652+05:30 [Info] Indexer Memory Deviation 80354383758 (74.8359G) (63.33%)
2023-09-01T13:38:26.652+05:30 [Info] Indexer Memory Utilization 2.3004
2023-09-01T13:38:26.652+05:30 [Info] Indexer CPU Mean 15.3333
2023-09-01T13:38:26.652+05:30 [Info] Indexer CPU Deviation 1.70 (11.08%)
2023-09-01T13:38:26.652+05:30 [Info] Indexer CPU Utilization 0.5679
2023-09-01T13:38:26.652+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:26.652+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:26.652+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:26.652+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:26.652+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:26.652+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:26.652+05:30 [Info] Indexer Data Size Mean 18082106963 (16.8403G)
2023-09-01T13:38:26.652+05:30 [Info] Indexer Data Size Deviation 25571960903 (23.8157G) (141.42%)
2023-09-01T13:38:26.652+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:26.652+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:26.652+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:26.652+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:26.652+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.652+05:30 [Info] Number of indexes: 13
2023-09-01T13:38:26.652+05:30 [Info] Number of indexers: 3
2023-09-01T13:38:26.652+05:30 [Info] Avg Index Size: 29281978949 (27.271G)
2023-09-01T13:38:26.652+05:30 [Info] Max Index Size: 73059719100 (68.0422G)
2023-09-01T13:38:26.652+05:30 [Info] Max Indexer Overhead: 30560384804 (28.4616G)
2023-09-01T13:38:26.652+05:30 [Info] Avg Index Cpu: 3.5385
2023-09-01T13:38:26.652+05:30 [Info] Max Index Cpu: 5.0000
2023-09-01T13:38:26.652+05:30 [Info] Num Estimation Run: 11
2023-09-01T13:38:26.652+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.652+05:30 [Info] Memory Quota 55158315092 (51.3702G)
2023-09-01T13:38:26.652+05:30 [Info] CPU Quota 27
2023-09-01T13:38:26.652+05:30 [Info] Max Cpu Utilization -1
2023-09-01T13:38:26.652+05:30 [Info] Max Memory Utilization -1
2023-09-01T13:38:26.652+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.652+05:30 [Info] Total iteration: 7010
2023-09-01T13:38:26.652+05:30 [Info] RandomSwap time: 0ns
2023-09-01T13:38:26.652+05:30 [Info] RandomSwap call: 0
2023-09-01T13:38:26.652+05:30 [Info] RandomSwap iteration: 0
2023-09-01T13:38:26.652+05:30 [Info] RandomMove time: 0ns
2023-09-01T13:38:26.652+05:30 [Info] RandomMove call: 0 (empty index 0)
2023-09-01T13:38:26.652+05:30 [Info] ExhaustMove time: 44ms
2023-09-01T13:38:26.652+05:30 [Info] ExhaustMove call: 7010
2023-09-01T13:38:26.652+05:30 [Info] ExhaustMove average time: 6us
2023-09-01T13:38:26.652+05:30 [Info] ExhaustSwap time: 0ns
2023-09-01T13:38:26.652+05:30 [Info] ExhaustSwap call: 0
2023-09-01T13:38:26.652+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.652+05:30 [Info] 
2023-09-01T13:38:26.652+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9000, nodeUUID:tempNodeUUID_2596996162, useLiveData:true
2023-09-01T13:38:26.652+05:30 [Info] Indexer total memory:267781689109 (249.391G), mem:237221304305 (220.93G), overhead:30560384804 (28.4616G), min:0 (0), data:0 (0) cpu:17.0000, io:0 (0), scan:0 drain:0 numIndexes:4
2023-09-01T13:38:26.652+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.37390490588193165 allowDDLDuringScaleup:true
2023-09-01T13:38:26.652+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.652+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.652+05:30 [Info] 		Index total memory:30961528072 (28.8352G), mem:27754610480 (25.8485G), overhead:3206917592 (2.98667G), min:5550922096 (5.1697G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.652+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.652+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.652+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.652+05:30 [Info] 		Index total memory:81043548208 (75.4777G), mem:73059719100 (68.0422G), overhead:7983829108 (7.43552G), min:14611943820 (13.6084G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.652+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.652+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.652+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.652+05:30 [Info] 		Index total memory:82702164802 (77.0224G), mem:72507837150 (67.5282G), overhead:10194327652 (9.49421G), min:14501567430 (13.5056G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.653+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.653+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.653+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.653+05:30 [Info] 		Index total memory:73074448027 (68.0559G), mem:63899137575 (59.5107G), overhead:9175310452 (8.54517G), min:12779827515 (11.9021G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.653+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.653+05:30 [Info] 
2023-09-01T13:38:26.653+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9001, nodeUUID:tempNodeUUID_4039455774, useLiveData:true
2023-09-01T13:38:26.653+05:30 [Info] Indexer total memory:111617261890 (103.952G), mem:95286106952 (88.7421G), overhead:16331154938 (15.2096G), min:54246320889 (50.5208G), data:54246320889 (50.5208G) cpu:13.0000, io:0 (0), scan:0 drain:0 numIndexes:5
2023-09-01T13:38:26.653+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.7501886306380632 allowDDLDuringScaleup:true
2023-09-01T13:38:26.653+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.653+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.653+05:30 [Info] 		Index total memory:8339546089 (7.76681G), mem:5550922096 (5.1697G), overhead:2788623993 (2.59711G), min:5550922096 (5.1697G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.653+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.653+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.653+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.653+05:30 [Info] 		Index total memory:21554403914 (20.0741G), mem:14611943820 (13.6084G), overhead:6942460094 (6.46567G), min:14611943820 (13.6084G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.653+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.653+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.653+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.653+05:30 [Info] 		Index total memory:13621658091 (12.6862G), mem:10444486052 (9.72719G), overhead:3177172039 (2.95897G), min:10444486052 (9.72719G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.653+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.653+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.653+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.653+05:30 [Info] 		Index total memory:13855332907 (12.9038G), mem:10432434095 (9.71596G), overhead:3422898812 (3.18782G), min:10432434095 (9.71596G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.653+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.653+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.653+05:30 [Info] 		Index name:idx1 0, bucket:bucket2, scope:, collection:, defnId:987654, instId:14531065497957473619, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.653+05:30 [Info] 		Index total memory:54246320889 (50.5208G), mem:54246320889 (50.5208G), overhead:0 (0), min:54246320889 (50.5208G), data:54246320889 (50.5208G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.653+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.653+05:30 [Info] 
2023-09-01T13:38:26.653+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9002, nodeUUID:tempNodeUUID_2854263694, useLiveData:true
2023-09-01T13:38:26.653+05:30 [Info] Indexer total memory:71601549512 (66.6841G), mem:48158315092 (44.8509G), overhead:23443234420 (21.8332G), min:0 (0), data:0 (0) cpu:16.0000, io:0 (0), scan:0 drain:0 numIndexes:4
2023-09-01T13:38:26.653+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.07590646348000514 allowDDLDuringScaleup:true
2023-09-01T13:38:26.653+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.653+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.653+05:30 [Info] 		Index total memory:23366200171 (21.7615G), mem:14501567430 (13.5056G), overhead:8864632741 (8.25583G), min:14501567430 (13.5056G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.653+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.653+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.653+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.653+05:30 [Info] 		Index total memory:20758358343 (19.3327G), mem:12779827515 (11.9021G), overhead:7978530828 (7.43059G), min:12779827515 (11.9021G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.653+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.653+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.653+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.653+05:30 [Info] 		Index total memory:13621658091 (12.6862G), mem:10444486052 (9.72719G), overhead:3177172039 (2.95897G), min:10444486052 (9.72719G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.653+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.653+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.653+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.653+05:30 [Info] 		Index total memory:13855332907 (12.9038G), mem:10432434095 (9.71596G), overhead:3422898812 (3.18782G), min:10432434095 (9.71596G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.653+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Index with 1 replica - 3 non-empty heterogenous nodes with 1 Scaled up - 1 SG
2023-09-01T13:38:26.655+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 1.0155476961840748
2023-09-01T13:38:26.734+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:26.734+05:30 [Info] Score: 0.28937833136341934
2023-09-01T13:38:26.734+05:30 [Info] variation: 0.5787566627268387
2023-09-01T13:38:26.734+05:30 [Info] ElapsedTime: 78ms
2023-09-01T13:38:26.734+05:30 [Info] ConvergenceTime: 75ms
2023-09-01T13:38:26.734+05:30 [Info] Iteration: 4300
2023-09-01T13:38:26.734+05:30 [Info] Move: 2627
2023-09-01T13:38:26.734+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.734+05:30 [Info] Using SAPlanner
2023-09-01T13:38:26.734+05:30 [Info] Indexer Memory Mean 144970682412 (135.014G)
2023-09-01T13:38:26.734+05:30 [Info] Indexer Memory Deviation 65295744085 (60.8114G) (45.04%)
2023-09-01T13:38:26.734+05:30 [Info] Indexer Memory Utilization 2.6283
2023-09-01T13:38:26.734+05:30 [Info] Indexer CPU Mean 15.3333
2023-09-01T13:38:26.734+05:30 [Info] Indexer CPU Deviation 1.70 (11.08%)
2023-09-01T13:38:26.734+05:30 [Info] Indexer CPU Utilization 0.5679
2023-09-01T13:38:26.734+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:26.734+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:26.734+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:26.734+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:26.734+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:26.734+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:26.734+05:30 [Info] Indexer Data Size Mean 36164213926 (33.6805G)
2023-09-01T13:38:26.734+05:30 [Info] Indexer Data Size Deviation 25571960903 (23.8157G) (70.71%)
2023-09-01T13:38:26.734+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:26.734+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:26.734+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:26.734+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:26.734+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.734+05:30 [Info] Number of indexes: 14
2023-09-01T13:38:26.734+05:30 [Info] Number of indexers: 3
2023-09-01T13:38:26.734+05:30 [Info] Avg Index Size: 31065146231 (28.9317G)
2023-09-01T13:38:26.734+05:30 [Info] Max Index Size: 73059719100 (68.0422G)
2023-09-01T13:38:26.734+05:30 [Info] Max Indexer Overhead: 30560384804 (28.4616G)
2023-09-01T13:38:26.734+05:30 [Info] Avg Index Cpu: 3.2857
2023-09-01T13:38:26.734+05:30 [Info] Max Index Cpu: 5.0000
2023-09-01T13:38:26.734+05:30 [Info] Num Estimation Run: 11
2023-09-01T13:38:26.734+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.734+05:30 [Info] Memory Quota 55158315092 (51.3702G)
2023-09-01T13:38:26.734+05:30 [Info] CPU Quota 27
2023-09-01T13:38:26.734+05:30 [Info] Max Cpu Utilization -1
2023-09-01T13:38:26.734+05:30 [Info] Max Memory Utilization -1
2023-09-01T13:38:26.734+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.734+05:30 [Info] Total iteration: 4310
2023-09-01T13:38:26.734+05:30 [Info] RandomSwap time: 0ns
2023-09-01T13:38:26.734+05:30 [Info] RandomSwap call: 0
2023-09-01T13:38:26.734+05:30 [Info] RandomSwap iteration: 0
2023-09-01T13:38:26.734+05:30 [Info] RandomMove time: 28ms
2023-09-01T13:38:26.735+05:30 [Info] RandomMove call: 4299 (empty index 0)
2023-09-01T13:38:26.735+05:30 [Info] RandomMove average time: 6us
2023-09-01T13:38:26.735+05:30 [Info] ExhaustMove time: 76us
2023-09-01T13:38:26.735+05:30 [Info] ExhaustMove call: 11
2023-09-01T13:38:26.735+05:30 [Info] ExhaustMove average time: 6us
2023-09-01T13:38:26.735+05:30 [Info] ExhaustSwap time: 0ns
2023-09-01T13:38:26.735+05:30 [Info] ExhaustSwap call: 0
2023-09-01T13:38:26.735+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.735+05:30 [Info] 
2023-09-01T13:38:26.735+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9000, nodeUUID:tempNodeUUID_2596996162, useLiveData:true
2023-09-01T13:38:26.735+05:30 [Info] Indexer total memory:267781689109 (249.391G), mem:237221304305 (220.93G), overhead:30560384804 (28.4616G), min:0 (0), data:0 (0) cpu:17.0000, io:0 (0), scan:0 drain:0 numIndexes:4
2023-09-01T13:38:26.735+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.32726796943638176 allowDDLDuringScaleup:true
2023-09-01T13:38:26.735+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.735+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.735+05:30 [Info] 		Index total memory:30961528072 (28.8352G), mem:27754610480 (25.8485G), overhead:3206917592 (2.98667G), min:5550922096 (5.1697G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.735+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.735+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.735+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.735+05:30 [Info] 		Index total memory:81043548208 (75.4777G), mem:73059719100 (68.0422G), overhead:7983829108 (7.43552G), min:14611943820 (13.6084G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.735+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.735+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.735+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.735+05:30 [Info] 		Index total memory:82702164802 (77.0224G), mem:72507837150 (67.5282G), overhead:10194327652 (9.49421G), min:14501567430 (13.5056G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.735+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.735+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.735+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.735+05:30 [Info] 		Index total memory:73074448027 (68.0559G), mem:63899137575 (59.5107G), overhead:9175310452 (8.54517G), min:12779827515 (11.9021G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.735+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.735+05:30 [Info] 
2023-09-01T13:38:26.735+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9001, nodeUUID:tempNodeUUID_4039455774, useLiveData:true
2023-09-01T13:38:26.735+05:30 [Info] Indexer total memory:111617261890 (103.952G), mem:95286106952 (88.7421G), overhead:16331154938 (15.2096G), min:54246320889 (50.5208G), data:54246320889 (50.5208G) cpu:13.0000, io:0 (0), scan:0 drain:0 numIndexes:5
2023-09-01T13:38:26.735+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.43145569209747264 allowDDLDuringScaleup:true
2023-09-01T13:38:26.735+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.735+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.735+05:30 [Info] 		Index total memory:8339546089 (7.76681G), mem:5550922096 (5.1697G), overhead:2788623993 (2.59711G), min:5550922096 (5.1697G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.735+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.735+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.735+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.735+05:30 [Info] 		Index total memory:21554403914 (20.0741G), mem:14611943820 (13.6084G), overhead:6942460094 (6.46567G), min:14611943820 (13.6084G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.735+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.735+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.735+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.735+05:30 [Info] 		Index total memory:13621658091 (12.6862G), mem:10444486052 (9.72719G), overhead:3177172039 (2.95897G), min:10444486052 (9.72719G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.735+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.735+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.735+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.735+05:30 [Info] 		Index total memory:13855332907 (12.9038G), mem:10432434095 (9.71596G), overhead:3422898812 (3.18782G), min:10432434095 (9.71596G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.735+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.735+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.735+05:30 [Info] 		Index name:idx1 0, bucket:bucket2, scope:, collection:, defnId:987654, instId:590072132871333004, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.735+05:30 [Info] 		Index total memory:54246320889 (50.5208G), mem:54246320889 (50.5208G), overhead:0 (0), min:54246320889 (50.5208G), data:54246320889 (50.5208G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.735+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.735+05:30 [Info] 
2023-09-01T13:38:26.735+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9002, nodeUUID:tempNodeUUID_2854263694, useLiveData:true
2023-09-01T13:38:26.735+05:30 [Info] Indexer total memory:125847870401 (117.205G), mem:102404635981 (95.3717G), overhead:23443234420 (21.8332G), min:54246320889 (50.5208G), data:54246320889 (50.5208G) cpu:16.0000, io:0 (0), scan:0 drain:0 numIndexes:5
2023-09-01T13:38:26.735+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.44127633846614567 allowDDLDuringScaleup:true
2023-09-01T13:38:26.735+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.735+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.735+05:30 [Info] 		Index total memory:23366200171 (21.7615G), mem:14501567430 (13.5056G), overhead:8864632741 (8.25583G), min:14501567430 (13.5056G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.735+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.735+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.735+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.735+05:30 [Info] 		Index total memory:20758358343 (19.3327G), mem:12779827515 (11.9021G), overhead:7978530828 (7.43059G), min:12779827515 (11.9021G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.735+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.735+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.735+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.735+05:30 [Info] 		Index total memory:13621658091 (12.6862G), mem:10444486052 (9.72719G), overhead:3177172039 (2.95897G), min:10444486052 (9.72719G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.735+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.735+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.735+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.735+05:30 [Info] 		Index total memory:13855332907 (12.9038G), mem:10432434095 (9.71596G), overhead:3422898812 (3.18782G), min:10432434095 (9.71596G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.736+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.736+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.736+05:30 [Info] 		Index name:idx1 0 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:590072132871333004, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.736+05:30 [Info] 		Index total memory:54246320889 (50.5208G), mem:54246320889 (50.5208G), overhead:0 (0), min:54246320889 (50.5208G), data:54246320889 (50.5208G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.736+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Index with 2 replicas - 3 non-empty heterogenous nodes with 1 Scaled up - 1 SG
2023-09-01T13:38:26.738+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.17181391709488233
2023-09-01T13:38:26.780+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:26.780+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=1.  Elapsed Time=41ms
2023-09-01T13:38:26.780+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.17181391709488233
2023-09-01T13:38:26.792+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:26.792+05:30 [Info] Score: 0.07993359954492428
2023-09-01T13:38:26.792+05:30 [Info] variation: 0.17181391709488233
2023-09-01T13:38:26.792+05:30 [Info] ElapsedTime: 11ms
2023-09-01T13:38:26.792+05:30 [Info] ConvergenceTime: 2us
2023-09-01T13:38:26.792+05:30 [Info] Iteration: 0
2023-09-01T13:38:26.792+05:30 [Info] Move: 0
2023-09-01T13:38:26.792+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.792+05:30 [Info] Using SAPlanner
2023-09-01T13:38:26.792+05:30 [Info] Indexer Memory Mean 264383397409 (246.226G)
2023-09-01T13:38:26.792+05:30 [Info] Indexer Memory Deviation 90849494247 (84.6102G) (34.36%)
2023-09-01T13:38:26.792+05:30 [Info] Indexer Memory Utilization 4.7932
2023-09-01T13:38:26.792+05:30 [Info] Indexer CPU Mean 15.3333
2023-09-01T13:38:26.792+05:30 [Info] Indexer CPU Deviation 1.70 (11.08%)
2023-09-01T13:38:26.792+05:30 [Info] Indexer CPU Utilization 0.5679
2023-09-01T13:38:26.792+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:26.792+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:26.792+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:26.792+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:26.792+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:26.792+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:26.792+05:30 [Info] Indexer Data Size Mean 155576928923 (144.892G)
2023-09-01T13:38:26.792+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2023-09-01T13:38:26.792+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:26.792+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:26.792+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:26.792+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:26.792+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.792+05:30 [Info] Number of indexes: 15
2023-09-01T13:38:26.792+05:30 [Info] Number of indexers: 3
2023-09-01T13:38:26.792+05:30 [Info] Avg Index Size: 21761293697 (20.2668G)
2023-09-01T13:38:26.792+05:30 [Info] Max Index Size: 73059719100 (68.0422G)
2023-09-01T13:38:26.792+05:30 [Info] Max Indexer Overhead: 30560384804 (28.4616G)
2023-09-01T13:38:26.792+05:30 [Info] Avg Index Cpu: 3.0667
2023-09-01T13:38:26.792+05:30 [Info] Max Index Cpu: 5.0000
2023-09-01T13:38:26.792+05:30 [Info] Num Estimation Run: 230
2023-09-01T13:38:26.792+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.792+05:30 [Info] Memory Quota 55158315092 (51.3702G)
2023-09-01T13:38:26.792+05:30 [Info] CPU Quota 27
2023-09-01T13:38:26.792+05:30 [Info] Max Cpu Utilization -1
2023-09-01T13:38:26.792+05:30 [Info] Max Memory Utilization -1
2023-09-01T13:38:26.792+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.792+05:30 [Info] Total iteration: 12870
2023-09-01T13:38:26.792+05:30 [Info] RandomSwap time: 0ns
2023-09-01T13:38:26.792+05:30 [Info] RandomSwap call: 0
2023-09-01T13:38:26.792+05:30 [Info] RandomSwap iteration: 0
2023-09-01T13:38:26.792+05:30 [Info] RandomMove time: 0ns
2023-09-01T13:38:26.792+05:30 [Info] RandomMove call: 0 (empty index 0)
2023-09-01T13:38:26.792+05:30 [Info] ExhaustMove time: 0ns
2023-09-01T13:38:26.792+05:30 [Info] ExhaustMove call: 0
2023-09-01T13:38:26.792+05:30 [Info] ExhaustSwap time: 0ns
2023-09-01T13:38:26.792+05:30 [Info] ExhaustSwap call: 0
2023-09-01T13:38:26.792+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.792+05:30 [Info] 
2023-09-01T13:38:26.792+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9000, nodeUUID:tempNodeUUID_2596996162, useLiveData:true
2023-09-01T13:38:26.792+05:30 [Info] Indexer total memory:423358618032 (394.283G), mem:392798233228 (365.822G), overhead:30560384804 (28.4616G), min:42699834774 (39.7673G), data:155576928923 (144.892G) cpu:17.0000, io:0 (0), scan:0 drain:0 numIndexes:5
2023-09-01T13:38:26.792+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.4971428895131053 allowDDLDuringScaleup:true
2023-09-01T13:38:26.792+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.793+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.793+05:30 [Info] 		Index total memory:30961528072 (28.8352G), mem:27754610480 (25.8485G), overhead:3206917592 (2.98667G), min:4995829886 (4.65273G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.793+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.793+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.793+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.793+05:30 [Info] 		Index total memory:81043548208 (75.4777G), mem:73059719100 (68.0422G), overhead:7983829108 (7.43552G), min:13150749438 (12.2476G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.793+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.793+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.793+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.793+05:30 [Info] 		Index total memory:82702164802 (77.0224G), mem:72507837150 (67.5282G), overhead:10194327652 (9.49421G), min:13051410687 (12.1551G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.793+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.793+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.793+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.793+05:30 [Info] 		Index total memory:73074448027 (68.0559G), mem:63899137575 (59.5107G), overhead:9175310452 (8.54517G), min:11501844763 (10.7119G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.793+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.793+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.793+05:30 [Info] 		Index name:idx1 0, bucket:bucket2, scope:, collection:, defnId:987654, instId:9563077627348951157, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.793+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.793+05:30 [Info] 		Index resident:0% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.793+05:30 [Info] 
2023-09-01T13:38:26.793+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9001, nodeUUID:tempNodeUUID_4039455774, useLiveData:true
2023-09-01T13:38:26.793+05:30 [Info] Indexer total memory:212947869924 (198.323G), mem:196616714986 (183.114G), overhead:16331154938 (15.2096G), min:36935807455 (34.3992G), data:155576928923 (144.892G) cpu:13.0000, io:0 (0), scan:0 drain:0 numIndexes:5
2023-09-01T13:38:26.793+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.34873605295368754 allowDDLDuringScaleup:true
2023-09-01T13:38:26.793+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.793+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.793+05:30 [Info] 		Index total memory:8339546089 (7.76681G), mem:5550922096 (5.1697G), overhead:2788623993 (2.59711G), min:4995829886 (4.65273G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.793+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.793+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.793+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.793+05:30 [Info] 		Index total memory:21554403914 (20.0741G), mem:14611943820 (13.6084G), overhead:6942460094 (6.46567G), min:13150749438 (12.2476G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.793+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.793+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.793+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.793+05:30 [Info] 		Index total memory:13621658091 (12.6862G), mem:10444486052 (9.72719G), overhead:3177172039 (2.95897G), min:9400037446 (8.75447G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.793+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.793+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.793+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.793+05:30 [Info] 		Index total memory:13855332907 (12.9038G), mem:10432434095 (9.71596G), overhead:3422898812 (3.18782G), min:9389190685 (8.74437G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.793+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.793+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.793+05:30 [Info] 		Index name:idx1 0 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:9563077627348951157, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.793+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.793+05:30 [Info] 		Index resident:0% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.793+05:30 [Info] 
2023-09-01T13:38:26.793+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9002, nodeUUID:tempNodeUUID_2854263694, useLiveData:true
2023-09-01T13:38:26.793+05:30 [Info] Indexer total memory:227178478435 (211.576G), mem:203735244015 (189.743G), overhead:23443234420 (21.8332G), min:43342483581 (40.3658G), data:155576928923 (144.892G) cpu:16.0000, io:0 (0), scan:0 drain:0 numIndexes:5
2023-09-01T13:38:26.793+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.3541210575332071 allowDDLDuringScaleup:true
2023-09-01T13:38:26.793+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.793+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.793+05:30 [Info] 		Index total memory:23366200171 (21.7615G), mem:14501567430 (13.5056G), overhead:8864632741 (8.25583G), min:13051410687 (12.1551G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.793+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.793+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.793+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.793+05:30 [Info] 		Index total memory:20758358343 (19.3327G), mem:12779827515 (11.9021G), overhead:7978530828 (7.43059G), min:11501844763 (10.7119G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.793+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.793+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.794+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.794+05:30 [Info] 		Index total memory:13621658091 (12.6862G), mem:10444486052 (9.72719G), overhead:3177172039 (2.95897G), min:9400037446 (8.75447G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.794+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.794+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.794+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.794+05:30 [Info] 		Index total memory:13855332907 (12.9038G), mem:10432434095 (9.71596G), overhead:3422898812 (3.18782G), min:9389190685 (8.74437G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.794+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.794+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.794+05:30 [Info] 		Index name:idx1 0 (replica 2), bucket:bucket2, scope:, collection:, defnId:987654, instId:9563077627348951157, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.794+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.794+05:30 [Info] 		Index resident:0% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Index With 1 Replica - 3 non-empty heterogenous nodes with 1 Scaled up - 2 SG - Placed on Scaled up node
2023-09-01T13:38:26.796+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.6856271841119315
2023-09-01T13:38:26.802+05:30 [Info] Planner::finalizing the solution as final solution is found.
2023-09-01T13:38:26.802+05:30 [Info] Score: 0.36056429186996203
2023-09-01T13:38:26.802+05:30 [Info] variation: 0.7211285837399241
2023-09-01T13:38:26.802+05:30 [Info] ElapsedTime: 5ms
2023-09-01T13:38:26.802+05:30 [Info] ConvergenceTime: 2ms
2023-09-01T13:38:26.802+05:30 [Info] Iteration: 1
2023-09-01T13:38:26.802+05:30 [Info] Move: 1
2023-09-01T13:38:26.802+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.802+05:30 [Info] Using SAPlanner
2023-09-01T13:38:26.802+05:30 [Info] Indexer Memory Mean 144970682412 (135.014G)
2023-09-01T13:38:26.802+05:30 [Info] Indexer Memory Deviation 106575253176 (99.2559G) (73.52%)
2023-09-01T13:38:26.802+05:30 [Info] Indexer Memory Utilization 2.6283
2023-09-01T13:38:26.802+05:30 [Info] Indexer CPU Mean 15.3333
2023-09-01T13:38:26.802+05:30 [Info] Indexer CPU Deviation 1.70 (11.08%)
2023-09-01T13:38:26.802+05:30 [Info] Indexer CPU Utilization 0.5679
2023-09-01T13:38:26.802+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:26.802+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:26.802+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:26.802+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:26.802+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:26.802+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:26.802+05:30 [Info] Indexer Data Size Mean 36164213926 (33.6805G)
2023-09-01T13:38:26.802+05:30 [Info] Indexer Data Size Deviation 25571960903 (23.8157G) (70.71%)
2023-09-01T13:38:26.802+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:26.802+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:26.802+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:26.802+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:26.802+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.802+05:30 [Info] Number of indexes: 14
2023-09-01T13:38:26.802+05:30 [Info] Number of indexers: 3
2023-09-01T13:38:26.802+05:30 [Info] Avg Index Size: 23315671818 (21.7144G)
2023-09-01T13:38:26.802+05:30 [Info] Max Index Size: 73059719100 (68.0422G)
2023-09-01T13:38:26.802+05:30 [Info] Max Indexer Overhead: 30560384804 (28.4616G)
2023-09-01T13:38:26.802+05:30 [Info] Avg Index Cpu: 3.2857
2023-09-01T13:38:26.802+05:30 [Info] Max Index Cpu: 5.0000
2023-09-01T13:38:26.802+05:30 [Info] Num Estimation Run: 230
2023-09-01T13:38:26.802+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.802+05:30 [Info] Memory Quota 55158315092 (51.3702G)
2023-09-01T13:38:26.802+05:30 [Info] CPU Quota 27
2023-09-01T13:38:26.802+05:30 [Info] Max Cpu Utilization -1
2023-09-01T13:38:26.802+05:30 [Info] Max Memory Utilization -1
2023-09-01T13:38:26.802+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.802+05:30 [Info] Total iteration: 523
2023-09-01T13:38:26.803+05:30 [Info] RandomSwap time: 0ns
2023-09-01T13:38:26.803+05:30 [Info] RandomSwap call: 0
2023-09-01T13:38:26.803+05:30 [Info] RandomSwap iteration: 0
2023-09-01T13:38:26.803+05:30 [Info] RandomMove time: 12us
2023-09-01T13:38:26.803+05:30 [Info] RandomMove call: 1 (empty index 0)
2023-09-01T13:38:26.803+05:30 [Info] RandomMove average time: 12us
2023-09-01T13:38:26.803+05:30 [Info] ExhaustMove time: 2ms
2023-09-01T13:38:26.803+05:30 [Info] ExhaustMove call: 220
2023-09-01T13:38:26.803+05:30 [Info] ExhaustMove average time: 9us
2023-09-01T13:38:26.803+05:30 [Info] ExhaustSwap time: 0ns
2023-09-01T13:38:26.803+05:30 [Info] ExhaustSwap call: 0
2023-09-01T13:38:26.803+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.803+05:30 [Info] 
2023-09-01T13:38:26.803+05:30 [Info] Indexer serverGroup:sg0, nodeId:127.0.0.1:9000, nodeUUID:tempNodeUUID_2596996162, useLiveData:true
2023-09-01T13:38:26.803+05:30 [Info] Indexer total memory:322028009998 (299.912G), mem:291467625194 (271.45G), overhead:30560384804 (28.4616G), min:54246320889 (50.5208G), data:54246320889 (50.5208G) cpu:17.0000, io:0 (0), scan:0 drain:0 numIndexes:5
2023-09-01T13:38:26.803+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:false usageRatio:0.7021056124497257 allowDDLDuringScaleup:true
2023-09-01T13:38:26.803+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.803+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.803+05:30 [Info] 		Index total memory:30961528072 (28.8352G), mem:27754610480 (25.8485G), overhead:3206917592 (2.98667G), min:5550922096 (5.1697G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.803+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.803+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.803+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.803+05:30 [Info] 		Index total memory:81043548208 (75.4777G), mem:73059719100 (68.0422G), overhead:7983829108 (7.43552G), min:14611943820 (13.6084G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.803+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.803+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.803+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.803+05:30 [Info] 		Index total memory:82702164802 (77.0224G), mem:72507837150 (67.5282G), overhead:10194327652 (9.49421G), min:14501567430 (13.5056G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.803+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.803+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.803+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.803+05:30 [Info] 		Index total memory:73074448027 (68.0559G), mem:63899137575 (59.5107G), overhead:9175310452 (8.54517G), min:12779827515 (11.9021G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.803+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.803+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.803+05:30 [Info] 		Index name:idx1 0, bucket:bucket2, scope:, collection:, defnId:987654, instId:4439159473378204875, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.803+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.803+05:30 [Info] 		Index resident:0% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.803+05:30 [Info] 
2023-09-01T13:38:26.803+05:30 [Info] Indexer serverGroup:sg1, nodeId:127.0.0.1:9001, nodeUUID:tempNodeUUID_4039455774, useLiveData:true
2023-09-01T13:38:26.803+05:30 [Info] Indexer total memory:57370941001 (53.4309G), mem:41039786063 (38.2213G), overhead:16331154938 (15.2096G), min:0 (0), data:0 (0) cpu:13.0000, io:0 (0), scan:0 drain:0 numIndexes:4
2023-09-01T13:38:26.803+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.05661804908412874 allowDDLDuringScaleup:true
2023-09-01T13:38:26.803+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.803+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.803+05:30 [Info] 		Index total memory:8339546089 (7.76681G), mem:5550922096 (5.1697G), overhead:2788623993 (2.59711G), min:5550922096 (5.1697G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.803+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.803+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.803+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.803+05:30 [Info] 		Index total memory:21554403914 (20.0741G), mem:14611943820 (13.6084G), overhead:6942460094 (6.46567G), min:14611943820 (13.6084G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.803+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.803+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.803+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.803+05:30 [Info] 		Index total memory:13621658091 (12.6862G), mem:10444486052 (9.72719G), overhead:3177172039 (2.95897G), min:10444486052 (9.72719G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.803+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.803+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.803+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.803+05:30 [Info] 		Index total memory:13855332907 (12.9038G), mem:10432434095 (9.71596G), overhead:3422898812 (3.18782G), min:10432434095 (9.71596G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.803+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.803+05:30 [Info] 
2023-09-01T13:38:26.803+05:30 [Info] Indexer serverGroup:sg1, nodeId:127.0.0.1:9002, nodeUUID:tempNodeUUID_2854263694, useLiveData:true
2023-09-01T13:38:26.803+05:30 [Info] Indexer total memory:125847870401 (117.205G), mem:102404635981 (95.3717G), overhead:23443234420 (21.8332G), min:54246320889 (50.5208G), data:54246320889 (50.5208G) cpu:16.0000, io:0 (0), scan:0 drain:0 numIndexes:5
2023-09-01T13:38:26.803+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.44127633846614567 allowDDLDuringScaleup:true
2023-09-01T13:38:26.803+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.803+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.803+05:30 [Info] 		Index total memory:23366200171 (21.7615G), mem:14501567430 (13.5056G), overhead:8864632741 (8.25583G), min:14501567430 (13.5056G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.803+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.803+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.803+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.803+05:30 [Info] 		Index total memory:20758358343 (19.3327G), mem:12779827515 (11.9021G), overhead:7978530828 (7.43059G), min:12779827515 (11.9021G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.803+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.803+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.803+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.803+05:30 [Info] 		Index total memory:13621658091 (12.6862G), mem:10444486052 (9.72719G), overhead:3177172039 (2.95897G), min:10444486052 (9.72719G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.803+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.803+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.803+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.803+05:30 [Info] 		Index total memory:13855332907 (12.9038G), mem:10432434095 (9.71596G), overhead:3422898812 (3.18782G), min:10432434095 (9.71596G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.803+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.803+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.803+05:30 [Info] 		Index name:idx1 0 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:4439159473378204875, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.803+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.804+05:30 [Info] 		Index resident:0% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Index With 1 Replica - 3 non-empty heterogenous nodes with 1 Scaled up - 2 SG - Placed on older nodes
2023-09-01T13:38:26.806+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.6917244207689
2023-09-01T13:38:26.811+05:30 [Info] Planner::finalizing the solution as final solution is found.
2023-09-01T13:38:26.811+05:30 [Info] Score: 0.28937833136341934
2023-09-01T13:38:26.811+05:30 [Info] variation: 0.5787566627268387
2023-09-01T13:38:26.811+05:30 [Info] ElapsedTime: 4ms
2023-09-01T13:38:26.811+05:30 [Info] ConvergenceTime: 2ms
2023-09-01T13:38:26.811+05:30 [Info] Iteration: 1
2023-09-01T13:38:26.811+05:30 [Info] Move: 1
2023-09-01T13:38:26.811+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.811+05:30 [Info] Using SAPlanner
2023-09-01T13:38:26.811+05:30 [Info] Indexer Memory Mean 144970682412 (135.014G)
2023-09-01T13:38:26.811+05:30 [Info] Indexer Memory Deviation 65295744085 (60.8114G) (45.04%)
2023-09-01T13:38:26.811+05:30 [Info] Indexer Memory Utilization 2.6283
2023-09-01T13:38:26.811+05:30 [Info] Indexer CPU Mean 15.3333
2023-09-01T13:38:26.811+05:30 [Info] Indexer CPU Deviation 1.70 (11.08%)
2023-09-01T13:38:26.811+05:30 [Info] Indexer CPU Utilization 0.5679
2023-09-01T13:38:26.811+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:26.811+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:26.811+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:26.811+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:26.811+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:26.811+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:26.811+05:30 [Info] Indexer Data Size Mean 36164213926 (33.6805G)
2023-09-01T13:38:26.811+05:30 [Info] Indexer Data Size Deviation 25571960903 (23.8157G) (70.71%)
2023-09-01T13:38:26.811+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:26.811+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:26.811+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:26.811+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:26.811+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.811+05:30 [Info] Number of indexes: 14
2023-09-01T13:38:26.811+05:30 [Info] Number of indexers: 3
2023-09-01T13:38:26.811+05:30 [Info] Avg Index Size: 23315671818 (21.7144G)
2023-09-01T13:38:26.811+05:30 [Info] Max Index Size: 73059719100 (68.0422G)
2023-09-01T13:38:26.811+05:30 [Info] Max Indexer Overhead: 30560384804 (28.4616G)
2023-09-01T13:38:26.811+05:30 [Info] Avg Index Cpu: 3.2857
2023-09-01T13:38:26.811+05:30 [Info] Max Index Cpu: 5.0000
2023-09-01T13:38:26.811+05:30 [Info] Num Estimation Run: 230
2023-09-01T13:38:26.811+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.811+05:30 [Info] Memory Quota 55158315092 (51.3702G)
2023-09-01T13:38:26.811+05:30 [Info] CPU Quota 27
2023-09-01T13:38:26.812+05:30 [Info] Max Cpu Utilization -1
2023-09-01T13:38:26.812+05:30 [Info] Max Memory Utilization -1
2023-09-01T13:38:26.812+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.812+05:30 [Info] Total iteration: 521
2023-09-01T13:38:26.812+05:30 [Info] RandomSwap time: 0ns
2023-09-01T13:38:26.812+05:30 [Info] RandomSwap call: 0
2023-09-01T13:38:26.812+05:30 [Info] RandomSwap iteration: 0
2023-09-01T13:38:26.812+05:30 [Info] RandomMove time: 12us
2023-09-01T13:38:26.812+05:30 [Info] RandomMove call: 1 (empty index 0)
2023-09-01T13:38:26.812+05:30 [Info] RandomMove average time: 12us
2023-09-01T13:38:26.812+05:30 [Info] ExhaustMove time: 1ms
2023-09-01T13:38:26.812+05:30 [Info] ExhaustMove call: 220
2023-09-01T13:38:26.812+05:30 [Info] ExhaustMove average time: 6us
2023-09-01T13:38:26.812+05:30 [Info] ExhaustSwap time: 0ns
2023-09-01T13:38:26.812+05:30 [Info] ExhaustSwap call: 0
2023-09-01T13:38:26.812+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.812+05:30 [Info] 
2023-09-01T13:38:26.812+05:30 [Info] Indexer serverGroup:sg1, nodeId:127.0.0.1:9000, nodeUUID:tempNodeUUID_2596996162, useLiveData:true
2023-09-01T13:38:26.812+05:30 [Info] Indexer total memory:267781689109 (249.391G), mem:237221304305 (220.93G), overhead:30560384804 (28.4616G), min:0 (0), data:0 (0) cpu:17.0000, io:0 (0), scan:0 drain:0 numIndexes:4
2023-09-01T13:38:26.812+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.32726796943638176 allowDDLDuringScaleup:true
2023-09-01T13:38:26.812+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.812+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.812+05:30 [Info] 		Index total memory:30961528072 (28.8352G), mem:27754610480 (25.8485G), overhead:3206917592 (2.98667G), min:5550922096 (5.1697G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.812+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.812+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.812+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.812+05:30 [Info] 		Index total memory:81043548208 (75.4777G), mem:73059719100 (68.0422G), overhead:7983829108 (7.43552G), min:14611943820 (13.6084G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.812+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.812+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.812+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.812+05:30 [Info] 		Index total memory:82702164802 (77.0224G), mem:72507837150 (67.5282G), overhead:10194327652 (9.49421G), min:14501567430 (13.5056G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.812+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.812+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.812+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.812+05:30 [Info] 		Index total memory:73074448027 (68.0559G), mem:63899137575 (59.5107G), overhead:9175310452 (8.54517G), min:12779827515 (11.9021G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.812+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.812+05:30 [Info] 
2023-09-01T13:38:26.812+05:30 [Info] Indexer serverGroup:sg1, nodeId:127.0.0.1:9001, nodeUUID:tempNodeUUID_4039455774, useLiveData:true
2023-09-01T13:38:26.812+05:30 [Info] Indexer total memory:111617261890 (103.952G), mem:95286106952 (88.7421G), overhead:16331154938 (15.2096G), min:54246320889 (50.5208G), data:54246320889 (50.5208G) cpu:13.0000, io:0 (0), scan:0 drain:0 numIndexes:5
2023-09-01T13:38:26.812+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.43145569209747264 allowDDLDuringScaleup:true
2023-09-01T13:38:26.812+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.812+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.812+05:30 [Info] 		Index total memory:8339546089 (7.76681G), mem:5550922096 (5.1697G), overhead:2788623993 (2.59711G), min:5550922096 (5.1697G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.812+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.812+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.812+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.812+05:30 [Info] 		Index total memory:21554403914 (20.0741G), mem:14611943820 (13.6084G), overhead:6942460094 (6.46567G), min:14611943820 (13.6084G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.812+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.812+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.812+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.812+05:30 [Info] 		Index total memory:13621658091 (12.6862G), mem:10444486052 (9.72719G), overhead:3177172039 (2.95897G), min:10444486052 (9.72719G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.812+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.813+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.813+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.813+05:30 [Info] 		Index total memory:13855332907 (12.9038G), mem:10432434095 (9.71596G), overhead:3422898812 (3.18782G), min:10432434095 (9.71596G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.813+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.813+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.813+05:30 [Info] 		Index name:idx1 0, bucket:bucket2, scope:, collection:, defnId:987654, instId:7522536440525951684, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.813+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.813+05:30 [Info] 		Index resident:0% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.813+05:30 [Info] 
2023-09-01T13:38:26.813+05:30 [Info] Indexer serverGroup:sg0, nodeId:127.0.0.1:9002, nodeUUID:tempNodeUUID_2854263694, useLiveData:true
2023-09-01T13:38:26.813+05:30 [Info] Indexer total memory:125847870401 (117.205G), mem:102404635981 (95.3717G), overhead:23443234420 (21.8332G), min:54246320889 (50.5208G), data:54246320889 (50.5208G) cpu:16.0000, io:0 (0), scan:0 drain:0 numIndexes:5
2023-09-01T13:38:26.813+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:false usageRatio:0.44127633846614567 allowDDLDuringScaleup:true
2023-09-01T13:38:26.813+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.813+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.813+05:30 [Info] 		Index total memory:23366200171 (21.7615G), mem:14501567430 (13.5056G), overhead:8864632741 (8.25583G), min:14501567430 (13.5056G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.813+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.813+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.813+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.813+05:30 [Info] 		Index total memory:20758358343 (19.3327G), mem:12779827515 (11.9021G), overhead:7978530828 (7.43059G), min:12779827515 (11.9021G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.813+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.813+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.813+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.813+05:30 [Info] 		Index total memory:13621658091 (12.6862G), mem:10444486052 (9.72719G), overhead:3177172039 (2.95897G), min:10444486052 (9.72719G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.813+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.813+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.813+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.813+05:30 [Info] 		Index total memory:13855332907 (12.9038G), mem:10432434095 (9.71596G), overhead:3422898812 (3.18782G), min:10432434095 (9.71596G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.813+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.813+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.813+05:30 [Info] 		Index name:idx1 0 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:7522536440525951684, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.813+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.813+05:30 [Info] 		Index resident:0% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Equivalent Index Without any replica - 3 non-empty heterogenous nodes with 1 Scaled up - 1 SG
2023-09-01T13:38:26.815+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.943232680780422
2023-09-01T13:38:26.817+05:30 [Info] Planner::finalizing the solution as final solution is found.
2023-09-01T13:38:26.817+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:26.817+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=1.  Elapsed Time=2ms
2023-09-01T13:38:26.817+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.943232680780422
2023-09-01T13:38:26.817+05:30 [Info] Planner::finalizing the solution as final solution is found.
2023-09-01T13:38:26.817+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:26.817+05:30 [Info] Score: 0.3533478080577597
2023-09-01T13:38:26.817+05:30 [Info] variation: 0.943232680780422
2023-09-01T13:38:26.817+05:30 [Info] ElapsedTime: 95us
2023-09-01T13:38:26.817+05:30 [Info] ConvergenceTime: 3us
2023-09-01T13:38:26.817+05:30 [Info] Iteration: 0
2023-09-01T13:38:26.817+05:30 [Info] Move: 0
2023-09-01T13:38:26.817+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.817+05:30 [Info] Using SAPlanner
2023-09-01T13:38:26.817+05:30 [Info] Indexer Memory Mean 17165606520 (15.9867G)
2023-09-01T13:38:26.817+05:30 [Info] Indexer Memory Deviation 8106488563 (7.54976G) (47.23%)
2023-09-01T13:38:26.817+05:30 [Info] Indexer Memory Utilization 1.9073
2023-09-01T13:38:26.817+05:30 [Info] Indexer CPU Mean 0.6667
2023-09-01T13:38:26.817+05:30 [Info] Indexer CPU Deviation 0.47 (70.71%)
2023-09-01T13:38:26.817+05:30 [Info] Indexer CPU Utilization 0.0833
2023-09-01T13:38:26.817+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:26.817+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:26.817+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:26.817+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:26.817+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:26.817+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:26.817+05:30 [Info] Indexer Data Size Mean 5165606520 (4.81085G)
2023-09-01T13:38:26.817+05:30 [Info] Indexer Data Size Deviation 7305270798 (6.80356G) (141.42%)
2023-09-01T13:38:26.817+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:26.817+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:26.817+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:26.817+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:26.818+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.818+05:30 [Info] Number of indexes: 4
2023-09-01T13:38:26.818+05:30 [Info] Number of indexers: 3
2023-09-01T13:38:26.818+05:30 [Info] Avg Index Size: 9000000000 (8.3819G)
2023-09-01T13:38:26.818+05:30 [Info] Max Index Size: 25000000000 (23.2831G)
2023-09-01T13:38:26.818+05:30 [Info] Max Indexer Overhead: 3000000000 (2.79397G)
2023-09-01T13:38:26.818+05:30 [Info] Avg Index Cpu: 0.7500
2023-09-01T13:38:26.818+05:30 [Info] Max Index Cpu: 1.0000
2023-09-01T13:38:26.818+05:30 [Info] Num Estimation Run: 208
2023-09-01T13:38:26.818+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.818+05:30 [Info] Memory Quota 9000000000 (8.3819G)
2023-09-01T13:38:26.818+05:30 [Info] CPU Quota 8
2023-09-01T13:38:26.818+05:30 [Info] Max Cpu Utilization -1
2023-09-01T13:38:26.818+05:30 [Info] Max Memory Utilization -1
2023-09-01T13:38:26.818+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.818+05:30 [Info] Total iteration: 209
2023-09-01T13:38:26.818+05:30 [Info] RandomSwap time: 0ns
2023-09-01T13:38:26.818+05:30 [Info] RandomSwap call: 0
2023-09-01T13:38:26.818+05:30 [Info] RandomSwap iteration: 0
2023-09-01T13:38:26.818+05:30 [Info] RandomMove time: 0ns
2023-09-01T13:38:26.818+05:30 [Info] RandomMove call: 0 (empty index 0)
2023-09-01T13:38:26.818+05:30 [Info] ExhaustMove time: 1ms
2023-09-01T13:38:26.818+05:30 [Info] ExhaustMove call: 209
2023-09-01T13:38:26.818+05:30 [Info] ExhaustMove average time: 5us
2023-09-01T13:38:26.818+05:30 [Info] ExhaustSwap time: 0ns
2023-09-01T13:38:26.818+05:30 [Info] ExhaustSwap call: 0
2023-09-01T13:38:26.818+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.818+05:30 [Info] 
2023-09-01T13:38:26.818+05:30 [Info] Indexer serverGroup:index 1, nodeId:127.0.0.1:9000, nodeUUID:c6fe7e12b609acc9767c7b671c908773, useLiveData:true
2023-09-01T13:38:26.818+05:30 [Info] Indexer total memory:28000000000 (26.077G), mem:25000000000 (23.2831G), overhead:3000000000 (2.79397G), min:4500000000 (4.19095G), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2023-09-01T13:38:26.818+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0.2912801242516189 allowDDLDuringScaleup:true
2023-09-01T13:38:26.818+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.818+05:30 [Info] 		Index name:equiv1 0, bucket:default, scope:, collection:, defnId:1062966916559205800, instId:1062966916559205800, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.818+05:30 [Info] 		Index total memory:28000000000 (26.077G), mem:25000000000 (23.2831G), overhead:3000000000 (2.79397G), min:4500000000 (4.19095G), data:0 (0) cpu:1.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.818+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.818+05:30 [Info] 
2023-09-01T13:38:26.818+05:30 [Info] Indexer serverGroup:index 1, nodeId:127.0.0.1:9001, nodeUUID:f18c1f3d3c1fabf2a63b51d06f536bf9, useLiveData:true
2023-09-01T13:38:26.818+05:30 [Info] Indexer total memory:8500000000 (7.91624G), mem:6000000000 (5.58794G), overhead:2500000000 (2.32831G), min:5400000000 (5.02914G), data:0 (0) cpu:1.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2023-09-01T13:38:26.818+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0.06990722982038854 allowDDLDuringScaleup:true
2023-09-01T13:38:26.818+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.818+05:30 [Info] 		Index name:equiv2 0, bucket:default, scope:, collection:, defnId:4650686066397790000, instId:4650686066397790000, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.818+05:30 [Info] 		Index total memory:8500000000 (7.91624G), mem:6000000000 (5.58794G), overhead:2500000000 (2.32831G), min:5400000000 (5.02914G), data:0 (0) cpu:1.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.818+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.818+05:30 [Info] 
2023-09-01T13:38:26.818+05:30 [Info] Indexer serverGroup:index 1, nodeId:127.0.0.1:9002, nodeUUID:d70e7e12b609acd0767c7b671c908889, useLiveData:true
2023-09-01T13:38:26.818+05:30 [Info] Indexer total memory:22496819560 (20.9518G), mem:20496819560 (19.0892G), overhead:2000000000 (1.86265G), min:4500000000 (4.19095G), data:15496819560 (14.4325G) cpu:1.0000, io:0 (0), scan:0 drain:0 numIndexes:2
2023-09-01T13:38:26.818+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0.8388126459279925 allowDDLDuringScaleup:true
2023-09-01T13:38:26.818+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.818+05:30 [Info] 		Index name:nonEquiv1 0, bucket:default, scope:, collection:, defnId:1062966916559205800, instId:1062966916559205800, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.818+05:30 [Info] 		Index total memory:7000000000 (6.51926G), mem:5000000000 (4.65661G), overhead:2000000000 (1.86265G), min:4500000000 (4.19095G), data:0 (0) cpu:1.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.818+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.818+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.818+05:30 [Info] 		Index name:newEquivIdx 0, bucket:default, scope:, collection:, defnId:987654, instId:424430646731530712, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.818+05:30 [Info] 		Index total memory:0 (0), mem:0 (0), overhead:0 (0), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.818+05:30 [Info] 		Index resident:0% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Equivalent Index With 1 Replica - 3 non-empty heterogenous nodes with 1 Scaled up - 1 SG - Use least loaded node
2023-09-01T13:38:26.820+05:30 [Warn] There are more equivalent index than available nodes.  Allow equivalent index of (default, , , newEquivIdx) to be replaced on same node.
2023-09-01T13:38:26.820+05:30 [Warn] Definition 987654 Instance 5203149409432743652 ReplicaId 0 partitionId 0 count 4 numLiveNode 3.
2023-09-01T13:38:26.820+05:30 [Warn] There are more equivalent index than available nodes.  Allow equivalent index of (default, , , newEquivIdx) to be replaced on same node.
2023-09-01T13:38:26.820+05:30 [Warn] Definition 987654 Instance 5203149409432743652 ReplicaId 1 partitionId 0 count 4 numLiveNode 3.
2023-09-01T13:38:26.820+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.6782899285088768
2023-09-01T13:38:26.882+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:26.882+05:30 [Info] Score: 0.25290029571859646
2023-09-01T13:38:26.882+05:30 [Info] variation: 0.5058005914371929
2023-09-01T13:38:26.882+05:30 [Info] ElapsedTime: 61ms
2023-09-01T13:38:26.882+05:30 [Info] ConvergenceTime: 58ms
2023-09-01T13:38:26.882+05:30 [Info] Iteration: 3800
2023-09-01T13:38:26.882+05:30 [Info] Move: 2409
2023-09-01T13:38:26.882+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.882+05:30 [Info] Using SAPlanner
2023-09-01T13:38:26.882+05:30 [Info] Indexer Memory Mean 17490430188 (16.2892G)
2023-09-01T13:38:26.882+05:30 [Info] Indexer Memory Deviation 5325738075 (4.95998G) (30.45%)
2023-09-01T13:38:26.882+05:30 [Info] Indexer Memory Utilization 1.9434
2023-09-01T13:38:26.882+05:30 [Info] Indexer CPU Mean 0.6667
2023-09-01T13:38:26.882+05:30 [Info] Indexer CPU Deviation 0.47 (70.71%)
2023-09-01T13:38:26.882+05:30 [Info] Indexer CPU Utilization 0.0833
2023-09-01T13:38:26.882+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:26.882+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:26.882+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:26.882+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:26.882+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:26.882+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:26.882+05:30 [Info] Indexer Data Size Mean 5490430188 (5.11336G)
2023-09-01T13:38:26.882+05:30 [Info] Indexer Data Size Deviation 3882320417 (3.61569G) (70.71%)
2023-09-01T13:38:26.882+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:26.882+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:26.882+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:26.882+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:26.882+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.882+05:30 [Info] Number of indexes: 5
2023-09-01T13:38:26.882+05:30 [Info] Number of indexers: 3
2023-09-01T13:38:26.882+05:30 [Info] Avg Index Size: 10494258112 (9.77354G)
2023-09-01T13:38:26.882+05:30 [Info] Max Index Size: 25000000000 (23.2831G)
2023-09-01T13:38:26.882+05:30 [Info] Max Indexer Overhead: 3000000000 (2.79397G)
2023-09-01T13:38:26.882+05:30 [Info] Avg Index Cpu: 0.6000
2023-09-01T13:38:26.882+05:30 [Info] Max Index Cpu: 1.0000
2023-09-01T13:38:26.882+05:30 [Info] Num Estimation Run: 7
2023-09-01T13:38:26.882+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.882+05:30 [Info] Memory Quota 9000000000 (8.3819G)
2023-09-01T13:38:26.882+05:30 [Info] CPU Quota 8
2023-09-01T13:38:26.882+05:30 [Info] Max Cpu Utilization -1
2023-09-01T13:38:26.882+05:30 [Info] Max Memory Utilization -1
2023-09-01T13:38:26.883+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.883+05:30 [Info] Total iteration: 3980
2023-09-01T13:38:26.883+05:30 [Info] RandomSwap time: 0ns
2023-09-01T13:38:26.883+05:30 [Info] RandomSwap call: 0
2023-09-01T13:38:26.883+05:30 [Info] RandomSwap iteration: 0
2023-09-01T13:38:26.883+05:30 [Info] RandomMove time: 20ms
2023-09-01T13:38:26.883+05:30 [Info] RandomMove call: 3799 (empty index 0)
2023-09-01T13:38:26.883+05:30 [Info] RandomMove average time: 5us
2023-09-01T13:38:26.883+05:30 [Info] ExhaustMove time: 7us
2023-09-01T13:38:26.883+05:30 [Info] ExhaustMove call: 1
2023-09-01T13:38:26.883+05:30 [Info] ExhaustMove average time: 7us
2023-09-01T13:38:26.883+05:30 [Info] ExhaustSwap time: 0ns
2023-09-01T13:38:26.883+05:30 [Info] ExhaustSwap call: 0
2023-09-01T13:38:26.883+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.883+05:30 [Info] 
2023-09-01T13:38:26.883+05:30 [Info] Indexer serverGroup:index 1, nodeId:127.0.0.1:9000, nodeUUID:c6fe7e12b609acc9767c7b671c908773, useLiveData:true
2023-09-01T13:38:26.883+05:30 [Info] Indexer total memory:28000000000 (26.077G), mem:25000000000 (23.2831G), overhead:3000000000 (2.79397G), min:0 (0), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2023-09-01T13:38:26.883+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0.2858706130298869 allowDDLDuringScaleup:true
2023-09-01T13:38:26.883+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.883+05:30 [Info] 		Index name:equiv1 0, bucket:default, scope:, collection:, defnId:1062966916559205800, instId:1062966916559205800, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.883+05:30 [Info] 		Index total memory:28000000000 (26.077G), mem:25000000000 (23.2831G), overhead:3000000000 (2.79397G), min:5000000000 (4.65661G), data:0 (0) cpu:1.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.883+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.883+05:30 [Info] 
2023-09-01T13:38:26.883+05:30 [Info] Indexer serverGroup:index 1, nodeId:127.0.0.1:9001, nodeUUID:f18c1f3d3c1fabf2a63b51d06f536bf9, useLiveData:true
2023-09-01T13:38:26.883+05:30 [Info] Indexer total memory:16735645282 (15.5863G), mem:14235645282 (13.258G), overhead:2500000000 (2.32831G), min:8235645282 (7.67004G), data:8235645282 (7.67004G) cpu:1.0000, io:0 (0), scan:0 drain:0 numIndexes:2
2023-09-01T13:38:26.883+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0.46278210574565426 allowDDLDuringScaleup:true
2023-09-01T13:38:26.883+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.883+05:30 [Info] 		Index name:equiv2 0, bucket:default, scope:, collection:, defnId:4650686066397790000, instId:4650686066397790000, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.883+05:30 [Info] 		Index total memory:8500000000 (7.91624G), mem:6000000000 (5.58794G), overhead:2500000000 (2.32831G), min:6000000000 (5.58794G), data:0 (0) cpu:1.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.883+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.883+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.883+05:30 [Info] 		Index name:newEquivIdx 0 (replica 1), bucket:default, scope:, collection:, defnId:987654, instId:5203149409432743652, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.883+05:30 [Info] 		Index total memory:8235645282 (7.67004G), mem:8235645282 (7.67004G), overhead:0 (0), min:8235645282 (7.67004G), data:8235645282 (7.67004G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.883+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:false pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.883+05:30 [Info] 
2023-09-01T13:38:26.883+05:30 [Info] Indexer serverGroup:index 1, nodeId:127.0.0.1:9002, nodeUUID:d70e7e12b609acd0767c7b671c908889, useLiveData:true
2023-09-01T13:38:26.883+05:30 [Info] Indexer total memory:15235645282 (14.1893G), mem:13235645282 (12.3267G), overhead:2000000000 (1.86265G), min:8235645282 (7.67004G), data:8235645282 (7.67004G) cpu:1.0000, io:0 (0), scan:0 drain:0 numIndexes:2
2023-09-01T13:38:26.883+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0.45134728122445883 allowDDLDuringScaleup:true
2023-09-01T13:38:26.883+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.883+05:30 [Info] 		Index name:nonEquiv1 0, bucket:default, scope:, collection:, defnId:1062966916559205800, instId:1062966916559205800, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.883+05:30 [Info] 		Index total memory:7000000000 (6.51926G), mem:5000000000 (4.65661G), overhead:2000000000 (1.86265G), min:5000000000 (4.65661G), data:0 (0) cpu:1.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.883+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.883+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.883+05:30 [Info] 		Index name:newEquivIdx 0, bucket:default, scope:, collection:, defnId:987654, instId:5203149409432743652, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.883+05:30 [Info] 		Index total memory:8235645282 (7.67004G), mem:8235645282 (7.67004G), overhead:0 (0), min:8235645282 (7.67004G), data:8235645282 (7.67004G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.883+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:false pendingCreate:false pendingDelete:false
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Equivalent Index With 1 Replica - 3 non-empty heterogenous nodes with 1 Scaled up - 1 SG - Use ScaleUp Node 
2023-09-01T13:38:26.885+05:30 [Warn] There are more equivalent index than available nodes.  Allow equivalent index of (default, , , newEquivIdx) to be replaced on same node.
2023-09-01T13:38:26.885+05:30 [Warn] Definition 987654 Instance 15997576541509632955 ReplicaId 1 partitionId 0 count 4 numLiveNode 3.
2023-09-01T13:38:26.885+05:30 [Warn] There are more equivalent index than available nodes.  Allow equivalent index of (default, , , newEquivIdx) to be replaced on same node.
2023-09-01T13:38:26.885+05:30 [Warn] Definition 987654 Instance 15997576541509632955 ReplicaId 0 partitionId 0 count 4 numLiveNode 3.
2023-09-01T13:38:26.885+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.6702149941767832
2023-09-01T13:38:26.945+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:26.945+05:30 [Info] Score: 0.25290029571859646
2023-09-01T13:38:26.945+05:30 [Info] variation: 0.5058005914371929
2023-09-01T13:38:26.945+05:30 [Info] ElapsedTime: 59ms
2023-09-01T13:38:26.945+05:30 [Info] ConvergenceTime: 57ms
2023-09-01T13:38:26.945+05:30 [Info] Iteration: 3800
2023-09-01T13:38:26.945+05:30 [Info] Move: 2409
2023-09-01T13:38:26.945+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.945+05:30 [Info] Using SAPlanner
2023-09-01T13:38:26.945+05:30 [Info] Indexer Memory Mean 17490430188 (16.2892G)
2023-09-01T13:38:26.945+05:30 [Info] Indexer Memory Deviation 5325738075 (4.95998G) (30.45%)
2023-09-01T13:38:26.945+05:30 [Info] Indexer Memory Utilization 1.9434
2023-09-01T13:38:26.945+05:30 [Info] Indexer CPU Mean 0.6667
2023-09-01T13:38:26.945+05:30 [Info] Indexer CPU Deviation 0.47 (70.71%)
2023-09-01T13:38:26.945+05:30 [Info] Indexer CPU Utilization 0.0833
2023-09-01T13:38:26.945+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:26.945+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:26.945+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:26.945+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:26.945+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:26.945+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:26.945+05:30 [Info] Indexer Data Size Mean 5490430188 (5.11336G)
2023-09-01T13:38:26.945+05:30 [Info] Indexer Data Size Deviation 3882320417 (3.61569G) (70.71%)
2023-09-01T13:38:26.945+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:26.945+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:26.945+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:26.945+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:26.945+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.945+05:30 [Info] Number of indexes: 5
2023-09-01T13:38:26.945+05:30 [Info] Number of indexers: 3
2023-09-01T13:38:26.945+05:30 [Info] Avg Index Size: 10494258112 (9.77354G)
2023-09-01T13:38:26.945+05:30 [Info] Max Index Size: 25000000000 (23.2831G)
2023-09-01T13:38:26.945+05:30 [Info] Max Indexer Overhead: 3000000000 (2.79397G)
2023-09-01T13:38:26.945+05:30 [Info] Avg Index Cpu: 0.6000
2023-09-01T13:38:26.945+05:30 [Info] Max Index Cpu: 1.0000
2023-09-01T13:38:26.945+05:30 [Info] Num Estimation Run: 7
2023-09-01T13:38:26.945+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.945+05:30 [Info] Memory Quota 9000000000 (8.3819G)
2023-09-01T13:38:26.945+05:30 [Info] CPU Quota 8
2023-09-01T13:38:26.945+05:30 [Info] Max Cpu Utilization -1
2023-09-01T13:38:26.945+05:30 [Info] Max Memory Utilization -1
2023-09-01T13:38:26.945+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.945+05:30 [Info] Total iteration: 3980
2023-09-01T13:38:26.945+05:30 [Info] RandomSwap time: 0ns
2023-09-01T13:38:26.945+05:30 [Info] RandomSwap call: 0
2023-09-01T13:38:26.945+05:30 [Info] RandomSwap iteration: 0
2023-09-01T13:38:26.945+05:30 [Info] RandomMove time: 19ms
2023-09-01T13:38:26.945+05:30 [Info] RandomMove call: 3799 (empty index 0)
2023-09-01T13:38:26.945+05:30 [Info] RandomMove average time: 5us
2023-09-01T13:38:26.945+05:30 [Info] ExhaustMove time: 7us
2023-09-01T13:38:26.945+05:30 [Info] ExhaustMove call: 1
2023-09-01T13:38:26.945+05:30 [Info] ExhaustMove average time: 7us
2023-09-01T13:38:26.945+05:30 [Info] ExhaustSwap time: 0ns
2023-09-01T13:38:26.945+05:30 [Info] ExhaustSwap call: 0
2023-09-01T13:38:26.945+05:30 [Info] ----------------------------------------
2023-09-01T13:38:26.945+05:30 [Info] 
2023-09-01T13:38:26.945+05:30 [Info] Indexer serverGroup:index 1, nodeId:127.0.0.1:9000, nodeUUID:c6fe7e12b609acc9767c7b671c908773, useLiveData:true
2023-09-01T13:38:26.945+05:30 [Info] Indexer total memory:28000000000 (26.077G), mem:25000000000 (23.2831G), overhead:3000000000 (2.79397G), min:0 (0), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2023-09-01T13:38:26.945+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0.2858706130298869 allowDDLDuringScaleup:true
2023-09-01T13:38:26.945+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.945+05:30 [Info] 		Index name:nonEquiv 0, bucket:default, scope:, collection:, defnId:1062966916559205800, instId:1062966916559205800, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.945+05:30 [Info] 		Index total memory:28000000000 (26.077G), mem:25000000000 (23.2831G), overhead:3000000000 (2.79397G), min:5000000000 (4.65661G), data:0 (0) cpu:1.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.945+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.945+05:30 [Info] 
2023-09-01T13:38:26.945+05:30 [Info] Indexer serverGroup:index 1, nodeId:127.0.0.1:9001, nodeUUID:f18c1f3d3c1fabf2a63b51d06f536bf9, useLiveData:true
2023-09-01T13:38:26.945+05:30 [Info] Indexer total memory:16735645282 (15.5863G), mem:14235645282 (13.258G), overhead:2500000000 (2.32831G), min:8235645282 (7.67004G), data:8235645282 (7.67004G) cpu:1.0000, io:0 (0), scan:0 drain:0 numIndexes:2
2023-09-01T13:38:26.945+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0.46278210574565426 allowDDLDuringScaleup:true
2023-09-01T13:38:26.945+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.945+05:30 [Info] 		Index name:equiv2 0, bucket:default, scope:, collection:, defnId:4650686066397790000, instId:4650686066397790000, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.945+05:30 [Info] 		Index total memory:8500000000 (7.91624G), mem:6000000000 (5.58794G), overhead:2500000000 (2.32831G), min:6000000000 (5.58794G), data:0 (0) cpu:1.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.945+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.945+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.945+05:30 [Info] 		Index name:newEquivIdx 0 (replica 1), bucket:default, scope:, collection:, defnId:987654, instId:15997576541509632955, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.945+05:30 [Info] 		Index total memory:8235645282 (7.67004G), mem:8235645282 (7.67004G), overhead:0 (0), min:8235645282 (7.67004G), data:8235645282 (7.67004G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.945+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:false pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.945+05:30 [Info] 
2023-09-01T13:38:26.945+05:30 [Info] Indexer serverGroup:index 1, nodeId:127.0.0.1:9002, nodeUUID:d70e7e12b609acd0767c7b671c908889, useLiveData:true
2023-09-01T13:38:26.945+05:30 [Info] Indexer total memory:15235645282 (14.1893G), mem:13235645282 (12.3267G), overhead:2000000000 (1.86265G), min:8235645282 (7.67004G), data:8235645282 (7.67004G) cpu:1.0000, io:0 (0), scan:0 drain:0 numIndexes:2
2023-09-01T13:38:26.945+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0.45134728122445883 allowDDLDuringScaleup:true
2023-09-01T13:38:26.945+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.945+05:30 [Info] 		Index name:equiv1 0, bucket:default, scope:, collection:, defnId:1062966916559205800, instId:1062966916559205800, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.945+05:30 [Info] 		Index total memory:7000000000 (6.51926G), mem:5000000000 (4.65661G), overhead:2000000000 (1.86265G), min:5000000000 (4.65661G), data:0 (0) cpu:1.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.945+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:26.945+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:26.945+05:30 [Info] 		Index name:newEquivIdx 0, bucket:default, scope:, collection:, defnId:987654, instId:15997576541509632955, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:26.945+05:30 [Info] 		Index total memory:8235645282 (7.67004G), mem:8235645282 (7.67004G), overhead:0 (0), min:8235645282 (7.67004G), data:8235645282 (7.67004G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:26.945+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:false pendingCreate:false pendingDelete:false
=== RUN   TestPlanDuringHeterogenousScaleup/SAPlanner/CreatePartitionedIndex
2023/09/01 13:38:26 -------------------------------------------
2023/09/01 13:38:26 Place Single Partitioned Index Instance - 3 non-empty heterogenous nodes with 1 Scaled up
2023-09-01T13:38:26.948+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.2964535944160216
2023-09-01T13:38:27.072+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:27.072+05:30 [Info] Score: 0.1784816836053567
2023-09-01T13:38:27.072+05:30 [Info] variation: 0.3569633672107134
2023-09-01T13:38:27.072+05:30 [Info] ElapsedTime: 124ms
2023-09-01T13:38:27.072+05:30 [Info] ConvergenceTime: 121ms
2023-09-01T13:38:27.072+05:30 [Info] Iteration: 5600
2023-09-01T13:38:27.073+05:30 [Info] Move: 3803
2023-09-01T13:38:27.073+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.073+05:30 [Info] Using SAPlanner
2023-09-01T13:38:27.073+05:30 [Info] Indexer Memory Mean 154201322596 (143.611G)
2023-09-01T13:38:27.073+05:30 [Info] Indexer Memory Deviation 82829246465 (77.1407G) (53.72%)
2023-09-01T13:38:27.073+05:30 [Info] Indexer Memory Utilization 2.7956
2023-09-01T13:38:27.073+05:30 [Info] Indexer CPU Mean 15.3333
2023-09-01T13:38:27.073+05:30 [Info] Indexer CPU Deviation 1.70 (11.08%)
2023-09-01T13:38:27.073+05:30 [Info] Indexer CPU Utilization 0.5679
2023-09-01T13:38:27.073+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:27.073+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:27.073+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:27.073+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:27.073+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:27.073+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:27.073+05:30 [Info] Indexer Data Size Mean 45394854109 (42.2773G)
2023-09-01T13:38:27.073+05:30 [Info] Indexer Data Size Deviation 8024752292 (7.47363G) (17.68%)
2023-09-01T13:38:27.073+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:27.073+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:27.073+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:27.073+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:27.073+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.073+05:30 [Info] Number of indexes: 20
2023-09-01T13:38:27.073+05:30 [Info] Number of indexers: 3
2023-09-01T13:38:27.073+05:30 [Info] Avg Index Size: 23130198389 (21.5417G)
2023-09-01T13:38:27.073+05:30 [Info] Max Index Size: 73059719100 (68.0422G)
2023-09-01T13:38:27.073+05:30 [Info] Max Indexer Overhead: 30560384804 (28.4616G)
2023-09-01T13:38:27.073+05:30 [Info] Avg Index Cpu: 2.3000
2023-09-01T13:38:27.073+05:30 [Info] Max Index Cpu: 5.0000
2023-09-01T13:38:27.073+05:30 [Info] Num Estimation Run: 22
2023-09-01T13:38:27.073+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.073+05:30 [Info] Memory Quota 55158315092 (51.3702G)
2023-09-01T13:38:27.073+05:30 [Info] CPU Quota 27
2023-09-01T13:38:27.073+05:30 [Info] Max Cpu Utilization -1
2023-09-01T13:38:27.073+05:30 [Info] Max Memory Utilization -1
2023-09-01T13:38:27.073+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.073+05:30 [Info] Total iteration: 9215
2023-09-01T13:38:27.073+05:30 [Info] RandomSwap time: 0ns
2023-09-01T13:38:27.073+05:30 [Info] RandomSwap call: 0
2023-09-01T13:38:27.073+05:30 [Info] RandomSwap iteration: 0
2023-09-01T13:38:27.073+05:30 [Info] RandomMove time: 33ms
2023-09-01T13:38:27.073+05:30 [Info] RandomMove call: 5599 (empty index 0)
2023-09-01T13:38:27.073+05:30 [Info] RandomMove average time: 6us
2023-09-01T13:38:27.073+05:30 [Info] ExhaustMove time: 8us
2023-09-01T13:38:27.073+05:30 [Info] ExhaustMove call: 1
2023-09-01T13:38:27.073+05:30 [Info] ExhaustMove average time: 8us
2023-09-01T13:38:27.073+05:30 [Info] ExhaustSwap time: 0ns
2023-09-01T13:38:27.073+05:30 [Info] ExhaustSwap call: 0
2023-09-01T13:38:27.073+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.073+05:30 [Info] 
2023-09-01T13:38:27.073+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9000, nodeUUID:tempNodeUUID_2596996162, useLiveData:true
2023-09-01T13:38:27.073+05:30 [Info] Indexer total memory:301827829691 (281.099G), mem:271267444887 (252.637G), overhead:30560384804 (28.4616G), min:34046140582 (31.7079G), data:34046140582 (31.7079G) cpu:17.0000, io:0 (0), scan:0 drain:0 numIndexes:6
2023-09-01T13:38:27.073+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.5018354321742201 allowDDLDuringScaleup:true
2023-09-01T13:38:27.073+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.073+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.073+05:30 [Info] 		Index total memory:30961528072 (28.8352G), mem:27754610480 (25.8485G), overhead:3206917592 (2.98667G), min:5550922096 (5.1697G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.073+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.073+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.073+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.073+05:30 [Info] 		Index total memory:81043548208 (75.4777G), mem:73059719100 (68.0422G), overhead:7983829108 (7.43552G), min:14611943820 (13.6084G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.073+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.073+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.073+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.073+05:30 [Info] 		Index total memory:82702164802 (77.0224G), mem:72507837150 (67.5282G), overhead:10194327652 (9.49421G), min:14501567430 (13.5056G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.073+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.073+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.073+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.073+05:30 [Info] 		Index total memory:73074448027 (68.0559G), mem:63899137575 (59.5107G), overhead:9175310452 (8.54517G), min:12779827515 (11.9021G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.073+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.073+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.073+05:30 [Info] 		Index name:idx1 6, bucket:bucket2, scope:, collection:, defnId:987654, instId:15622911138812185632, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.073+05:30 [Info] 		Index total memory:17023070291 (15.854G), mem:17023070291 (15.854G), overhead:0 (0), min:17023070291 (15.854G), data:17023070291 (15.854G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.073+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.073+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.073+05:30 [Info] 		Index name:idx1 1, bucket:bucket2, scope:, collection:, defnId:987654, instId:15622911138812185632, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.073+05:30 [Info] 		Index total memory:17023070291 (15.854G), mem:17023070291 (15.854G), overhead:0 (0), min:17023070291 (15.854G), data:17023070291 (15.854G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.073+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.073+05:30 [Info] 
2023-09-01T13:38:27.073+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9001, nodeUUID:tempNodeUUID_4039455774, useLiveData:true
2023-09-01T13:38:27.073+05:30 [Info] Indexer total memory:108440151874 (100.993G), mem:92108996936 (85.7832G), overhead:16331154938 (15.2096G), min:51069210873 (47.5619G), data:51069210873 (47.5619G) cpu:13.0000, io:0 (0), scan:0 drain:0 numIndexes:7
2023-09-01T13:38:27.073+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.3444658974194678 allowDDLDuringScaleup:true
2023-09-01T13:38:27.073+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.073+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.074+05:30 [Info] 		Index total memory:8339546089 (7.76681G), mem:5550922096 (5.1697G), overhead:2788623993 (2.59711G), min:5550922096 (5.1697G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.074+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.074+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.074+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.074+05:30 [Info] 		Index total memory:21554403914 (20.0741G), mem:14611943820 (13.6084G), overhead:6942460094 (6.46567G), min:14611943820 (13.6084G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.074+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.074+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.074+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.074+05:30 [Info] 		Index total memory:13621658091 (12.6862G), mem:10444486052 (9.72719G), overhead:3177172039 (2.95897G), min:10444486052 (9.72719G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.074+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.074+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.074+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.074+05:30 [Info] 		Index total memory:13855332907 (12.9038G), mem:10432434095 (9.71596G), overhead:3422898812 (3.18782G), min:10432434095 (9.71596G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.074+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.074+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.074+05:30 [Info] 		Index name:idx1 5, bucket:bucket2, scope:, collection:, defnId:987654, instId:15622911138812185632, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.074+05:30 [Info] 		Index total memory:17023070291 (15.854G), mem:17023070291 (15.854G), overhead:0 (0), min:17023070291 (15.854G), data:17023070291 (15.854G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.074+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.074+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.074+05:30 [Info] 		Index name:idx1 3, bucket:bucket2, scope:, collection:, defnId:987654, instId:15622911138812185632, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.074+05:30 [Info] 		Index total memory:17023070291 (15.854G), mem:17023070291 (15.854G), overhead:0 (0), min:17023070291 (15.854G), data:17023070291 (15.854G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.074+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.074+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.074+05:30 [Info] 		Index name:idx1 7, bucket:bucket2, scope:, collection:, defnId:987654, instId:15622911138812185632, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.074+05:30 [Info] 		Index total memory:17023070291 (15.854G), mem:17023070291 (15.854G), overhead:0 (0), min:17023070291 (15.854G), data:17023070291 (15.854G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.074+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.074+05:30 [Info] 
2023-09-01T13:38:27.074+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9002, nodeUUID:tempNodeUUID_2854263694, useLiveData:true
2023-09-01T13:38:27.074+05:30 [Info] Indexer total memory:122670760385 (114.246G), mem:99227525965 (92.4128G), overhead:23443234420 (21.8332G), min:51069210873 (47.5619G), data:51069210873 (47.5619G) cpu:16.0000, io:0 (0), scan:0 drain:0 numIndexes:7
2023-09-01T13:38:27.074+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.35369867040631203 allowDDLDuringScaleup:true
2023-09-01T13:38:27.074+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.074+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.074+05:30 [Info] 		Index total memory:23366200171 (21.7615G), mem:14501567430 (13.5056G), overhead:8864632741 (8.25583G), min:14501567430 (13.5056G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.074+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.074+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.074+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.074+05:30 [Info] 		Index total memory:20758358343 (19.3327G), mem:12779827515 (11.9021G), overhead:7978530828 (7.43059G), min:12779827515 (11.9021G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.074+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.074+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.074+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.074+05:30 [Info] 		Index total memory:13621658091 (12.6862G), mem:10444486052 (9.72719G), overhead:3177172039 (2.95897G), min:10444486052 (9.72719G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.074+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.074+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.074+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.074+05:30 [Info] 		Index total memory:13855332907 (12.9038G), mem:10432434095 (9.71596G), overhead:3422898812 (3.18782G), min:10432434095 (9.71596G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.074+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.074+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.074+05:30 [Info] 		Index name:idx1 0, bucket:bucket2, scope:, collection:, defnId:987654, instId:15622911138812185632, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.074+05:30 [Info] 		Index total memory:17023070291 (15.854G), mem:17023070291 (15.854G), overhead:0 (0), min:17023070291 (15.854G), data:17023070291 (15.854G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.074+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.074+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.074+05:30 [Info] 		Index name:idx1 4, bucket:bucket2, scope:, collection:, defnId:987654, instId:15622911138812185632, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.074+05:30 [Info] 		Index total memory:17023070291 (15.854G), mem:17023070291 (15.854G), overhead:0 (0), min:17023070291 (15.854G), data:17023070291 (15.854G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.074+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.074+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.074+05:30 [Info] 		Index name:idx1 2, bucket:bucket2, scope:, collection:, defnId:987654, instId:15622911138812185632, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.074+05:30 [Info] 		Index total memory:17023070291 (15.854G), mem:17023070291 (15.854G), overhead:0 (0), min:17023070291 (15.854G), data:17023070291 (15.854G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.074+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023/09/01 13:38:27 -------------------------------------------
2023/09/01 13:38:27 Place Single Partitioned Index Instance - 3 non-empty heterogenous nodes with 2 Scaled up
2023-09-01T13:38:27.077+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.21948680330955667
2023-09-01T13:38:27.245+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:27.245+05:30 [Info] Score: 0.13739736706800032
2023-09-01T13:38:27.245+05:30 [Info] variation: 0.27479473413600064
2023-09-01T13:38:27.245+05:30 [Info] ElapsedTime: 168ms
2023-09-01T13:38:27.245+05:30 [Info] ConvergenceTime: 165ms
2023-09-01T13:38:27.245+05:30 [Info] Iteration: 6900
2023-09-01T13:38:27.245+05:30 [Info] Move: 4874
2023-09-01T13:38:27.245+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.245+05:30 [Info] Using SAPlanner
2023-09-01T13:38:27.245+05:30 [Info] Indexer Memory Mean 208747525973 (194.411G)
2023-09-01T13:38:27.245+05:30 [Info] Indexer Memory Deviation 77823744009 (72.479G) (37.28%)
2023-09-01T13:38:27.245+05:30 [Info] Indexer Memory Utilization 3.7845
2023-09-01T13:38:27.245+05:30 [Info] Indexer CPU Mean 15.3333
2023-09-01T13:38:27.245+05:30 [Info] Indexer CPU Deviation 1.70 (11.08%)
2023-09-01T13:38:27.245+05:30 [Info] Indexer CPU Utilization 0.5679
2023-09-01T13:38:27.245+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:27.245+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:27.245+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:27.245+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:27.245+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:27.245+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:27.245+05:30 [Info] Indexer Data Size Mean 45221342736 (42.1157G)
2023-09-01T13:38:27.245+05:30 [Info] Indexer Data Size Deviation 7994079525 (7.44507G) (17.68%)
2023-09-01T13:38:27.245+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:27.245+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:27.246+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:27.246+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:27.246+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.246+05:30 [Info] Number of indexes: 20
2023-09-01T13:38:27.246+05:30 [Info] Number of indexers: 3
2023-09-01T13:38:27.246+05:30 [Info] Avg Index Size: 31312128896 (29.1617G)
2023-09-01T13:38:27.246+05:30 [Info] Max Index Size: 73059719100 (68.0422G)
2023-09-01T13:38:27.246+05:30 [Info] Max Indexer Overhead: 81655774690 (76.0479G)
2023-09-01T13:38:27.246+05:30 [Info] Avg Index Cpu: 2.3000
2023-09-01T13:38:27.246+05:30 [Info] Max Index Cpu: 5.0000
2023-09-01T13:38:27.246+05:30 [Info] Num Estimation Run: 23
2023-09-01T13:38:27.246+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.246+05:30 [Info] Memory Quota 55158315092 (51.3702G)
2023-09-01T13:38:27.246+05:30 [Info] CPU Quota 27
2023-09-01T13:38:27.246+05:30 [Info] Max Cpu Utilization -1
2023-09-01T13:38:27.246+05:30 [Info] Max Memory Utilization -1
2023-09-01T13:38:27.246+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.246+05:30 [Info] Total iteration: 10567
2023-09-01T13:38:27.246+05:30 [Info] RandomSwap time: 0ns
2023-09-01T13:38:27.246+05:30 [Info] RandomSwap call: 0
2023-09-01T13:38:27.246+05:30 [Info] RandomSwap iteration: 0
2023-09-01T13:38:27.246+05:30 [Info] RandomMove time: 45ms
2023-09-01T13:38:27.246+05:30 [Info] RandomMove call: 6899 (empty index 0)
2023-09-01T13:38:27.246+05:30 [Info] RandomMove average time: 6us
2023-09-01T13:38:27.246+05:30 [Info] ExhaustMove time: 8us
2023-09-01T13:38:27.246+05:30 [Info] ExhaustMove call: 1
2023-09-01T13:38:27.246+05:30 [Info] ExhaustMove average time: 8us
2023-09-01T13:38:27.246+05:30 [Info] ExhaustSwap time: 0ns
2023-09-01T13:38:27.246+05:30 [Info] ExhaustSwap call: 0
2023-09-01T13:38:27.246+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.246+05:30 [Info] 
2023-09-01T13:38:27.246+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9000, nodeUUID:tempNodeUUID_2596996162, useLiveData:true
2023-09-01T13:38:27.246+05:30 [Info] Indexer total memory:301697696161 (280.978G), mem:271137311357 (252.516G), overhead:30560384804 (28.4616G), min:33916007052 (31.5867G), data:33916007052 (31.5867G) cpu:17.0000, io:0 (0), scan:0 drain:0 numIndexes:6
2023-09-01T13:38:27.246+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.4097753531140166 allowDDLDuringScaleup:true
2023-09-01T13:38:27.246+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.246+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.246+05:30 [Info] 		Index total memory:30961528072 (28.8352G), mem:27754610480 (25.8485G), overhead:3206917592 (2.98667G), min:5550922096 (5.1697G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.246+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.246+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.246+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.246+05:30 [Info] 		Index total memory:81043548208 (75.4777G), mem:73059719100 (68.0422G), overhead:7983829108 (7.43552G), min:14611943820 (13.6084G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.246+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.246+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.246+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.246+05:30 [Info] 		Index total memory:82702164802 (77.0224G), mem:72507837150 (67.5282G), overhead:10194327652 (9.49421G), min:14501567430 (13.5056G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.246+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.246+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.246+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.246+05:30 [Info] 		Index total memory:73074448027 (68.0559G), mem:63899137575 (59.5107G), overhead:9175310452 (8.54517G), min:12779827515 (11.9021G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.246+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.246+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.246+05:30 [Info] 		Index name:idx1 7, bucket:bucket2, scope:, collection:, defnId:987654, instId:8947911854475630760, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.246+05:30 [Info] 		Index total memory:16958003526 (15.7934G), mem:16958003526 (15.7934G), overhead:0 (0), min:16958003526 (15.7934G), data:16958003526 (15.7934G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.246+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.246+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.246+05:30 [Info] 		Index name:idx1 1, bucket:bucket2, scope:, collection:, defnId:987654, instId:8947911854475630760, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.246+05:30 [Info] 		Index total memory:16958003526 (15.7934G), mem:16958003526 (15.7934G), overhead:0 (0), min:16958003526 (15.7934G), data:16958003526 (15.7934G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.246+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.246+05:30 [Info] 
2023-09-01T13:38:27.247+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9001, nodeUUID:tempNodeUUID_4039455774, useLiveData:true
2023-09-01T13:38:27.247+05:30 [Info] Indexer total memory:337728715583 (314.534G), mem:256072940893 (238.487G), overhead:81655774690 (76.0479G), min:50874010578 (47.3801G), data:50874010578 (47.3801G) cpu:13.0000, io:0 (0), scan:0 drain:0 numIndexes:7
2023-09-01T13:38:27.247+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.470342252272453 allowDDLDuringScaleup:true
2023-09-01T13:38:27.247+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.247+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.247+05:30 [Info] 		Index total memory:30961528072 (28.8352G), mem:27754610480 (25.8485G), overhead:3206917592 (2.98667G), min:5550922096 (5.1697G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.247+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.247+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.247+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.247+05:30 [Info] 		Index total memory:81043548208 (75.4777G), mem:73059719100 (68.0422G), overhead:7983829108 (7.43552G), min:14611943820 (13.6084G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.247+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.247+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.247+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.247+05:30 [Info] 		Index total memory:55876178105 (52.0387G), mem:52222430260 (48.6359G), overhead:3653747845 (3.40282G), min:10444486052 (9.72719G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.247+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.247+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.247+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.247+05:30 [Info] 		Index total memory:56098504109 (52.2458G), mem:52162170475 (48.5798G), overhead:3936333634 (3.666G), min:10432434095 (9.71596G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.247+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.247+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.247+05:30 [Info] 		Index name:idx1 4, bucket:bucket2, scope:, collection:, defnId:987654, instId:8947911854475630760, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.247+05:30 [Info] 		Index total memory:16958003526 (15.7934G), mem:16958003526 (15.7934G), overhead:0 (0), min:16958003526 (15.7934G), data:16958003526 (15.7934G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.247+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.247+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.247+05:30 [Info] 		Index name:idx1 0, bucket:bucket2, scope:, collection:, defnId:987654, instId:8947911854475630760, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.247+05:30 [Info] 		Index total memory:16958003526 (15.7934G), mem:16958003526 (15.7934G), overhead:0 (0), min:16958003526 (15.7934G), data:16958003526 (15.7934G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.247+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.247+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.247+05:30 [Info] 		Index name:idx1 3, bucket:bucket2, scope:, collection:, defnId:987654, instId:8947911854475630760, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.247+05:30 [Info] 		Index total memory:16958003526 (15.7934G), mem:16958003526 (15.7934G), overhead:0 (0), min:16958003526 (15.7934G), data:16958003526 (15.7934G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.247+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.247+05:30 [Info] 
2023-09-01T13:38:27.247+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9002, nodeUUID:tempNodeUUID_2854263694, useLiveData:true
2023-09-01T13:38:27.247+05:30 [Info] Indexer total memory:122475560090 (114.064G), mem:99032325670 (92.231G), overhead:23443234420 (21.8332G), min:50874010578 (47.3801G), data:50874010578 (47.3801G) cpu:16.0000, io:0 (0), scan:0 drain:0 numIndexes:7
2023-09-01T13:38:27.247+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.3198823946135304 allowDDLDuringScaleup:true
2023-09-01T13:38:27.247+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.247+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.247+05:30 [Info] 		Index total memory:23366200171 (21.7615G), mem:14501567430 (13.5056G), overhead:8864632741 (8.25583G), min:14501567430 (13.5056G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.247+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.247+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.247+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.247+05:30 [Info] 		Index total memory:20758358343 (19.3327G), mem:12779827515 (11.9021G), overhead:7978530828 (7.43059G), min:12779827515 (11.9021G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.247+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.247+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.247+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.247+05:30 [Info] 		Index total memory:13621658091 (12.6862G), mem:10444486052 (9.72719G), overhead:3177172039 (2.95897G), min:10444486052 (9.72719G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.247+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.247+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.247+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.247+05:30 [Info] 		Index total memory:13855332907 (12.9038G), mem:10432434095 (9.71596G), overhead:3422898812 (3.18782G), min:10432434095 (9.71596G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.247+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.247+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.247+05:30 [Info] 		Index name:idx1 2, bucket:bucket2, scope:, collection:, defnId:987654, instId:8947911854475630760, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.247+05:30 [Info] 		Index total memory:16958003526 (15.7934G), mem:16958003526 (15.7934G), overhead:0 (0), min:16958003526 (15.7934G), data:16958003526 (15.7934G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.247+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.247+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.247+05:30 [Info] 		Index name:idx1 5, bucket:bucket2, scope:, collection:, defnId:987654, instId:8947911854475630760, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.247+05:30 [Info] 		Index total memory:16958003526 (15.7934G), mem:16958003526 (15.7934G), overhead:0 (0), min:16958003526 (15.7934G), data:16958003526 (15.7934G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.247+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.247+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.247+05:30 [Info] 		Index name:idx1 6, bucket:bucket2, scope:, collection:, defnId:987654, instId:8947911854475630760, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.247+05:30 [Info] 		Index total memory:16958003526 (15.7934G), mem:16958003526 (15.7934G), overhead:0 (0), min:16958003526 (15.7934G), data:16958003526 (15.7934G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.247+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023/09/01 13:38:27 -------------------------------------------
2023/09/01 13:38:27 Place Paritioned Index with 1 replica - 3 non-empty heterogenous nodes with 1 Scaled up
2023-09-01T13:38:27.250+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.47836546373712163
2023-09-01T13:38:27.453+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:27.453+05:30 [Info] Score: 0.16324776491504728
2023-09-01T13:38:27.453+05:30 [Info] variation: 0.32649552983009456
2023-09-01T13:38:27.453+05:30 [Info] ElapsedTime: 203ms
2023-09-01T13:38:27.453+05:30 [Info] ConvergenceTime: 200ms
2023-09-01T13:38:27.453+05:30 [Info] Iteration: 8600
2023-09-01T13:38:27.453+05:30 [Info] Move: 4598
2023-09-01T13:38:27.454+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.454+05:30 [Info] Using SAPlanner
2023-09-01T13:38:27.454+05:30 [Info] Indexer Memory Mean 157055841798 (146.27G)
2023-09-01T13:38:27.454+05:30 [Info] Indexer Memory Deviation 88674154216 (82.5842G) (56.46%)
2023-09-01T13:38:27.454+05:30 [Info] Indexer Memory Utilization 2.8474
2023-09-01T13:38:27.454+05:30 [Info] Indexer CPU Mean 15.3333
2023-09-01T13:38:27.454+05:30 [Info] Indexer CPU Deviation 1.70 (11.08%)
2023-09-01T13:38:27.454+05:30 [Info] Indexer CPU Utilization 0.5679
2023-09-01T13:38:27.454+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:27.454+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:27.454+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:27.454+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:27.454+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:27.454+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:27.454+05:30 [Info] Indexer Data Size Mean 48249373312 (44.9357G)
2023-09-01T13:38:27.454+05:30 [Info] Indexer Data Size Deviation 4264682382 (3.97179G) (8.84%)
2023-09-01T13:38:27.454+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:27.454+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:27.454+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:27.454+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:27.454+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.454+05:30 [Info] Number of indexes: 28
2023-09-01T13:38:27.454+05:30 [Info] Number of indexers: 3
2023-09-01T13:38:27.454+05:30 [Info] Avg Index Size: 16827411621 (15.6717G)
2023-09-01T13:38:27.454+05:30 [Info] Max Index Size: 73059719100 (68.0422G)
2023-09-01T13:38:27.454+05:30 [Info] Max Indexer Overhead: 30560384804 (28.4616G)
2023-09-01T13:38:27.454+05:30 [Info] Avg Index Cpu: 1.6429
2023-09-01T13:38:27.454+05:30 [Info] Max Index Cpu: 5.0000
2023-09-01T13:38:27.454+05:30 [Info] Num Estimation Run: 28
2023-09-01T13:38:27.454+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.454+05:30 [Info] Memory Quota 55158315092 (51.3702G)
2023-09-01T13:38:27.454+05:30 [Info] CPU Quota 27
2023-09-01T13:38:27.454+05:30 [Info] Max Cpu Utilization -1
2023-09-01T13:38:27.454+05:30 [Info] Max Memory Utilization -1
2023-09-01T13:38:27.454+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.454+05:30 [Info] Total iteration: 13766
2023-09-01T13:38:27.454+05:30 [Info] RandomSwap time: 0ns
2023-09-01T13:38:27.454+05:30 [Info] RandomSwap call: 0
2023-09-01T13:38:27.454+05:30 [Info] RandomSwap iteration: 0
2023-09-01T13:38:27.454+05:30 [Info] RandomMove time: 57ms
2023-09-01T13:38:27.454+05:30 [Info] RandomMove call: 8599 (empty index 0)
2023-09-01T13:38:27.454+05:30 [Info] RandomMove average time: 6us
2023-09-01T13:38:27.454+05:30 [Info] ExhaustMove time: 10us
2023-09-01T13:38:27.454+05:30 [Info] ExhaustMove call: 1
2023-09-01T13:38:27.454+05:30 [Info] ExhaustMove average time: 10us
2023-09-01T13:38:27.454+05:30 [Info] ExhaustSwap time: 0ns
2023-09-01T13:38:27.454+05:30 [Info] ExhaustSwap call: 0
2023-09-01T13:38:27.454+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.454+05:30 [Info] 
2023-09-01T13:38:27.454+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9000, nodeUUID:tempNodeUUID_2596996162, useLiveData:true
2023-09-01T13:38:27.454+05:30 [Info] Indexer total memory:313015476589 (291.518G), mem:282455091785 (263.057G), overhead:30560384804 (28.4616G), min:45233787480 (42.1272G), data:45233787480 (42.1272G) cpu:17.0000, io:0 (0), scan:0 drain:0 numIndexes:9
2023-09-01T13:38:27.454+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.5471874698199197 allowDDLDuringScaleup:true
2023-09-01T13:38:27.454+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.454+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.454+05:30 [Info] 		Index total memory:30961528072 (28.8352G), mem:27754610480 (25.8485G), overhead:3206917592 (2.98667G), min:5550922096 (5.1697G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.454+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.454+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.454+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.454+05:30 [Info] 		Index total memory:81043548208 (75.4777G), mem:73059719100 (68.0422G), overhead:7983829108 (7.43552G), min:14611943820 (13.6084G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.454+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.454+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.454+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.454+05:30 [Info] 		Index total memory:82702164802 (77.0224G), mem:72507837150 (67.5282G), overhead:10194327652 (9.49421G), min:14501567430 (13.5056G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.454+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.454+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.454+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.454+05:30 [Info] 		Index total memory:73074448027 (68.0559G), mem:63899137575 (59.5107G), overhead:9175310452 (8.54517G), min:12779827515 (11.9021G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.454+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.454+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.454+05:30 [Info] 		Index name:idx1 1 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:4342093409795388516, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.454+05:30 [Info] 		Index total memory:9046757496 (8.42545G), mem:9046757496 (8.42545G), overhead:0 (0), min:9046757496 (8.42545G), data:9046757496 (8.42545G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.454+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.454+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.454+05:30 [Info] 		Index name:idx1 4, bucket:bucket2, scope:, collection:, defnId:987654, instId:4342093409795388516, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.454+05:30 [Info] 		Index total memory:9046757496 (8.42545G), mem:9046757496 (8.42545G), overhead:0 (0), min:9046757496 (8.42545G), data:9046757496 (8.42545G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.454+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.454+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.454+05:30 [Info] 		Index name:idx1 3, bucket:bucket2, scope:, collection:, defnId:987654, instId:4342093409795388516, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.454+05:30 [Info] 		Index total memory:9046757496 (8.42545G), mem:9046757496 (8.42545G), overhead:0 (0), min:9046757496 (8.42545G), data:9046757496 (8.42545G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.454+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.454+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.454+05:30 [Info] 		Index name:idx1 7 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:4342093409795388516, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.454+05:30 [Info] 		Index total memory:9046757496 (8.42545G), mem:9046757496 (8.42545G), overhead:0 (0), min:9046757496 (8.42545G), data:9046757496 (8.42545G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.454+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.454+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.454+05:30 [Info] 		Index name:idx1 6 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:4342093409795388516, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.454+05:30 [Info] 		Index total memory:9046757496 (8.42545G), mem:9046757496 (8.42545G), overhead:0 (0), min:9046757496 (8.42545G), data:9046757496 (8.42545G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.454+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.454+05:30 [Info] 
2023-09-01T13:38:27.454+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9001, nodeUUID:tempNodeUUID_4039455774, useLiveData:true
2023-09-01T13:38:27.454+05:30 [Info] Indexer total memory:111651485977 (103.984G), mem:95320331039 (88.774G), overhead:16331154938 (15.2096G), min:54280544976 (50.5527G), data:54280544976 (50.5527G) cpu:13.0000, io:0 (0), scan:0 drain:0 numIndexes:10
2023-09-01T13:38:27.454+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.346383999407285 allowDDLDuringScaleup:true
2023-09-01T13:38:27.454+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.454+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.454+05:30 [Info] 		Index total memory:8339546089 (7.76681G), mem:5550922096 (5.1697G), overhead:2788623993 (2.59711G), min:5550922096 (5.1697G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.454+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.455+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.455+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.455+05:30 [Info] 		Index total memory:21554403914 (20.0741G), mem:14611943820 (13.6084G), overhead:6942460094 (6.46567G), min:14611943820 (13.6084G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.455+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.455+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.455+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.455+05:30 [Info] 		Index total memory:13621658091 (12.6862G), mem:10444486052 (9.72719G), overhead:3177172039 (2.95897G), min:10444486052 (9.72719G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.455+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.455+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.455+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.455+05:30 [Info] 		Index total memory:13855332907 (12.9038G), mem:10432434095 (9.71596G), overhead:3422898812 (3.18782G), min:10432434095 (9.71596G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.455+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.455+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.455+05:30 [Info] 		Index name:idx1 0 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:4342093409795388516, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.455+05:30 [Info] 		Index total memory:9046757496 (8.42545G), mem:9046757496 (8.42545G), overhead:0 (0), min:9046757496 (8.42545G), data:9046757496 (8.42545G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.455+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.455+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.455+05:30 [Info] 		Index name:idx1 5, bucket:bucket2, scope:, collection:, defnId:987654, instId:4342093409795388516, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.455+05:30 [Info] 		Index total memory:9046757496 (8.42545G), mem:9046757496 (8.42545G), overhead:0 (0), min:9046757496 (8.42545G), data:9046757496 (8.42545G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.455+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.455+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.455+05:30 [Info] 		Index name:idx1 2, bucket:bucket2, scope:, collection:, defnId:987654, instId:4342093409795388516, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.455+05:30 [Info] 		Index total memory:9046757496 (8.42545G), mem:9046757496 (8.42545G), overhead:0 (0), min:9046757496 (8.42545G), data:9046757496 (8.42545G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.455+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.455+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.455+05:30 [Info] 		Index name:idx1 7, bucket:bucket2, scope:, collection:, defnId:987654, instId:4342093409795388516, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.455+05:30 [Info] 		Index total memory:9046757496 (8.42545G), mem:9046757496 (8.42545G), overhead:0 (0), min:9046757496 (8.42545G), data:9046757496 (8.42545G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.455+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.455+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.455+05:30 [Info] 		Index name:idx1 6, bucket:bucket2, scope:, collection:, defnId:987654, instId:4342093409795388516, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.455+05:30 [Info] 		Index total memory:9046757496 (8.42545G), mem:9046757496 (8.42545G), overhead:0 (0), min:9046757496 (8.42545G), data:9046757496 (8.42545G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.455+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.455+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.455+05:30 [Info] 		Index name:idx1 4 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:4342093409795388516, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.455+05:30 [Info] 		Index total memory:9046757496 (8.42545G), mem:9046757496 (8.42545G), overhead:0 (0), min:9046757496 (8.42545G), data:9046757496 (8.42545G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.455+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.455+05:30 [Info] 
2023-09-01T13:38:27.455+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9002, nodeUUID:tempNodeUUID_2854263694, useLiveData:true
2023-09-01T13:38:27.455+05:30 [Info] Indexer total memory:116835336992 (108.811G), mem:93392102572 (86.9782G), overhead:23443234420 (21.8332G), min:45233787480 (42.1272G), data:45233787480 (42.1272G) cpu:16.0000, io:0 (0), scan:0 drain:0 numIndexes:9
2023-09-01T13:38:27.455+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.3064285307727953 allowDDLDuringScaleup:true
2023-09-01T13:38:27.455+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.455+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.455+05:30 [Info] 		Index total memory:23366200171 (21.7615G), mem:14501567430 (13.5056G), overhead:8864632741 (8.25583G), min:14501567430 (13.5056G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.455+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.455+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.455+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.455+05:30 [Info] 		Index total memory:20758358343 (19.3327G), mem:12779827515 (11.9021G), overhead:7978530828 (7.43059G), min:12779827515 (11.9021G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.455+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.455+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.455+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.455+05:30 [Info] 		Index total memory:13621658091 (12.6862G), mem:10444486052 (9.72719G), overhead:3177172039 (2.95897G), min:10444486052 (9.72719G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.455+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.455+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.455+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.455+05:30 [Info] 		Index total memory:13855332907 (12.9038G), mem:10432434095 (9.71596G), overhead:3422898812 (3.18782G), min:10432434095 (9.71596G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.455+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.455+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.455+05:30 [Info] 		Index name:idx1 0, bucket:bucket2, scope:, collection:, defnId:987654, instId:4342093409795388516, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.455+05:30 [Info] 		Index total memory:9046757496 (8.42545G), mem:9046757496 (8.42545G), overhead:0 (0), min:9046757496 (8.42545G), data:9046757496 (8.42545G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.455+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.455+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.455+05:30 [Info] 		Index name:idx1 5 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:4342093409795388516, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.455+05:30 [Info] 		Index total memory:9046757496 (8.42545G), mem:9046757496 (8.42545G), overhead:0 (0), min:9046757496 (8.42545G), data:9046757496 (8.42545G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.455+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.455+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.455+05:30 [Info] 		Index name:idx1 2 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:4342093409795388516, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.455+05:30 [Info] 		Index total memory:9046757496 (8.42545G), mem:9046757496 (8.42545G), overhead:0 (0), min:9046757496 (8.42545G), data:9046757496 (8.42545G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.455+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.455+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.455+05:30 [Info] 		Index name:idx1 3 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:4342093409795388516, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.455+05:30 [Info] 		Index total memory:9046757496 (8.42545G), mem:9046757496 (8.42545G), overhead:0 (0), min:9046757496 (8.42545G), data:9046757496 (8.42545G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.455+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.455+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.455+05:30 [Info] 		Index name:idx1 1, bucket:bucket2, scope:, collection:, defnId:987654, instId:4342093409795388516, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.455+05:30 [Info] 		Index total memory:9046757496 (8.42545G), mem:9046757496 (8.42545G), overhead:0 (0), min:9046757496 (8.42545G), data:9046757496 (8.42545G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.455+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023/09/01 13:38:27 -------------------------------------------
2023/09/01 13:38:27 Place Paritioned Index with 1 replica - 3 non-empty heterogenous nodes with 2 Scaled up
2023-09-01T13:38:27.458+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.17893635394493998
2023-09-01T13:38:27.591+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:27.591+05:30 [Info] Score: 0.11992230442867714
2023-09-01T13:38:27.591+05:30 [Info] variation: 0.23984460885735429
2023-09-01T13:38:27.591+05:30 [Info] ElapsedTime: 132ms
2023-09-01T13:38:27.591+05:30 [Info] ConvergenceTime: 129ms
2023-09-01T13:38:27.591+05:30 [Info] Iteration: 6200
2023-09-01T13:38:27.591+05:30 [Info] Move: 3878
2023-09-01T13:38:27.591+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.591+05:30 [Info] Using SAPlanner
2023-09-01T13:38:27.591+05:30 [Info] Indexer Memory Mean 202458793706 (188.554G)
2023-09-01T13:38:27.591+05:30 [Info] Indexer Memory Deviation 79222302130 (73.7815G) (39.13%)
2023-09-01T13:38:27.591+05:30 [Info] Indexer Memory Utilization 3.6705
2023-09-01T13:38:27.591+05:30 [Info] Indexer CPU Mean 15.3333
2023-09-01T13:38:27.591+05:30 [Info] Indexer CPU Deviation 1.70 (11.08%)
2023-09-01T13:38:27.591+05:30 [Info] Indexer CPU Utilization 0.5679
2023-09-01T13:38:27.591+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:27.591+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:27.591+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:27.591+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:27.591+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:27.591+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:27.591+05:30 [Info] Indexer Data Size Mean 38932610469 (36.2588G)
2023-09-01T13:38:27.591+05:30 [Info] Indexer Data Size Deviation 3441189109 (3.20486G) (8.84%)
2023-09-01T13:38:27.591+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:27.591+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:27.591+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:27.591+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:27.591+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.591+05:30 [Info] Number of indexes: 28
2023-09-01T13:38:27.591+05:30 [Info] Number of indexers: 3
2023-09-01T13:38:27.591+05:30 [Info] Avg Index Size: 21692013611 (20.2023G)
2023-09-01T13:38:27.591+05:30 [Info] Max Index Size: 73059719100 (68.0422G)
2023-09-01T13:38:27.591+05:30 [Info] Max Indexer Overhead: 81655774690 (76.0479G)
2023-09-01T13:38:27.591+05:30 [Info] Avg Index Cpu: 1.6429
2023-09-01T13:38:27.591+05:30 [Info] Max Index Cpu: 5.0000
2023-09-01T13:38:27.591+05:30 [Info] Num Estimation Run: 31
2023-09-01T13:38:27.591+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.591+05:30 [Info] Memory Quota 55158315092 (51.3702G)
2023-09-01T13:38:27.591+05:30 [Info] CPU Quota 27
2023-09-01T13:38:27.591+05:30 [Info] Max Cpu Utilization -1
2023-09-01T13:38:27.591+05:30 [Info] Max Memory Utilization -1
2023-09-01T13:38:27.591+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.591+05:30 [Info] Total iteration: 7383
2023-09-01T13:38:27.591+05:30 [Info] RandomSwap time: 0ns
2023-09-01T13:38:27.591+05:30 [Info] RandomSwap call: 0
2023-09-01T13:38:27.591+05:30 [Info] RandomSwap iteration: 0
2023-09-01T13:38:27.591+05:30 [Info] RandomMove time: 41ms
2023-09-01T13:38:27.591+05:30 [Info] RandomMove call: 6197 (empty index 0)
2023-09-01T13:38:27.591+05:30 [Info] RandomMove average time: 6us
2023-09-01T13:38:27.591+05:30 [Info] ExhaustMove time: 49us
2023-09-01T13:38:27.591+05:30 [Info] ExhaustMove call: 5
2023-09-01T13:38:27.591+05:30 [Info] ExhaustMove average time: 9us
2023-09-01T13:38:27.591+05:30 [Info] ExhaustSwap time: 0ns
2023-09-01T13:38:27.591+05:30 [Info] ExhaustSwap call: 0
2023-09-01T13:38:27.591+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.591+05:30 [Info] 
2023-09-01T13:38:27.591+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9000, nodeUUID:tempNodeUUID_2596996162, useLiveData:true
2023-09-01T13:38:27.591+05:30 [Info] Indexer total memory:304281011424 (283.384G), mem:273720626620 (254.922G), overhead:30560384804 (28.4616G), min:36499322315 (33.9926G), data:36499322315 (33.9926G) cpu:17.0000, io:0 (0), scan:0 drain:0 numIndexes:9
2023-09-01T13:38:27.591+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.45789638200806565 allowDDLDuringScaleup:true
2023-09-01T13:38:27.591+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.591+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.591+05:30 [Info] 		Index total memory:30961528072 (28.8352G), mem:27754610480 (25.8485G), overhead:3206917592 (2.98667G), min:5550922096 (5.1697G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.591+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.591+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.591+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.591+05:30 [Info] 		Index total memory:81043548208 (75.4777G), mem:73059719100 (68.0422G), overhead:7983829108 (7.43552G), min:14611943820 (13.6084G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.591+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.591+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.591+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.591+05:30 [Info] 		Index total memory:82702164802 (77.0224G), mem:72507837150 (67.5282G), overhead:10194327652 (9.49421G), min:14501567430 (13.5056G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.591+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.591+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.591+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.591+05:30 [Info] 		Index total memory:73074448027 (68.0559G), mem:63899137575 (59.5107G), overhead:9175310452 (8.54517G), min:12779827515 (11.9021G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.591+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.591+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.591+05:30 [Info] 		Index name:idx1 5 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:11972398092819331361, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.592+05:30 [Info] 		Index total memory:7299864463 (6.79853G), mem:7299864463 (6.79853G), overhead:0 (0), min:7299864463 (6.79853G), data:7299864463 (6.79853G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.592+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.592+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.592+05:30 [Info] 		Index name:idx1 3 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:11972398092819331361, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.592+05:30 [Info] 		Index total memory:7299864463 (6.79853G), mem:7299864463 (6.79853G), overhead:0 (0), min:7299864463 (6.79853G), data:7299864463 (6.79853G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.592+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.592+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.592+05:30 [Info] 		Index name:idx1 2, bucket:bucket2, scope:, collection:, defnId:987654, instId:11972398092819331361, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.592+05:30 [Info] 		Index total memory:7299864463 (6.79853G), mem:7299864463 (6.79853G), overhead:0 (0), min:7299864463 (6.79853G), data:7299864463 (6.79853G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.592+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.592+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.592+05:30 [Info] 		Index name:idx1 7 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:11972398092819331361, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.592+05:30 [Info] 		Index total memory:7299864463 (6.79853G), mem:7299864463 (6.79853G), overhead:0 (0), min:7299864463 (6.79853G), data:7299864463 (6.79853G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.592+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.592+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.592+05:30 [Info] 		Index name:idx1 6 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:11972398092819331361, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.592+05:30 [Info] 		Index total memory:7299864463 (6.79853G), mem:7299864463 (6.79853G), overhead:0 (0), min:7299864463 (6.79853G), data:7299864463 (6.79853G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.592+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.592+05:30 [Info] 
2023-09-01T13:38:27.592+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9001, nodeUUID:tempNodeUUID_4039455774, useLiveData:true
2023-09-01T13:38:27.592+05:30 [Info] Indexer total memory:323354027320 (301.147G), mem:241698252630 (225.099G), overhead:81655774690 (76.0479G), min:36499322315 (33.9926G), data:36499322315 (33.9926G) cpu:13.0000, io:0 (0), scan:0 drain:0 numIndexes:9
2023-09-01T13:38:27.592+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.4262629089405577 allowDDLDuringScaleup:true
2023-09-01T13:38:27.592+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.592+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.592+05:30 [Info] 		Index total memory:30961528072 (28.8352G), mem:27754610480 (25.8485G), overhead:3206917592 (2.98667G), min:5550922096 (5.1697G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.592+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.592+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.592+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.592+05:30 [Info] 		Index total memory:81043548208 (75.4777G), mem:73059719100 (68.0422G), overhead:7983829108 (7.43552G), min:14611943820 (13.6084G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.592+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.592+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.592+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.592+05:30 [Info] 		Index total memory:55876178105 (52.0387G), mem:52222430260 (48.6359G), overhead:3653747845 (3.40282G), min:10444486052 (9.72719G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.592+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.592+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.592+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.592+05:30 [Info] 		Index total memory:56098504109 (52.2458G), mem:52162170475 (48.5798G), overhead:3936333634 (3.666G), min:10432434095 (9.71596G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.592+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.592+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.592+05:30 [Info] 		Index name:idx1 4 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:11972398092819331361, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.592+05:30 [Info] 		Index total memory:7299864463 (6.79853G), mem:7299864463 (6.79853G), overhead:0 (0), min:7299864463 (6.79853G), data:7299864463 (6.79853G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.592+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.592+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.592+05:30 [Info] 		Index name:idx1 0 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:11972398092819331361, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.592+05:30 [Info] 		Index total memory:7299864463 (6.79853G), mem:7299864463 (6.79853G), overhead:0 (0), min:7299864463 (6.79853G), data:7299864463 (6.79853G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.592+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.592+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.592+05:30 [Info] 		Index name:idx1 5, bucket:bucket2, scope:, collection:, defnId:987654, instId:11972398092819331361, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.592+05:30 [Info] 		Index total memory:7299864463 (6.79853G), mem:7299864463 (6.79853G), overhead:0 (0), min:7299864463 (6.79853G), data:7299864463 (6.79853G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.592+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.592+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.592+05:30 [Info] 		Index name:idx1 7, bucket:bucket2, scope:, collection:, defnId:987654, instId:11972398092819331361, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.592+05:30 [Info] 		Index total memory:7299864463 (6.79853G), mem:7299864463 (6.79853G), overhead:0 (0), min:7299864463 (6.79853G), data:7299864463 (6.79853G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.592+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.592+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.592+05:30 [Info] 		Index name:idx1 1 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:11972398092819331361, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.592+05:30 [Info] 		Index total memory:7299864463 (6.79853G), mem:7299864463 (6.79853G), overhead:0 (0), min:7299864463 (6.79853G), data:7299864463 (6.79853G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.592+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.592+05:30 [Info] 
2023-09-01T13:38:27.592+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9002, nodeUUID:tempNodeUUID_2854263694, useLiveData:true
2023-09-01T13:38:27.592+05:30 [Info] Indexer total memory:115400736290 (107.475G), mem:91957501870 (85.6421G), overhead:23443234420 (21.8332G), min:43799186778 (40.7912G), data:43799186778 (40.7912G) cpu:16.0000, io:0 (0), scan:0 drain:0 numIndexes:10
2023-09-01T13:38:27.592+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.3158407090513767 allowDDLDuringScaleup:true
2023-09-01T13:38:27.592+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.592+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.592+05:30 [Info] 		Index total memory:23366200171 (21.7615G), mem:14501567430 (13.5056G), overhead:8864632741 (8.25583G), min:14501567430 (13.5056G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.592+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.592+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.592+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.592+05:30 [Info] 		Index total memory:20758358343 (19.3327G), mem:12779827515 (11.9021G), overhead:7978530828 (7.43059G), min:12779827515 (11.9021G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.592+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.592+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.592+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.592+05:30 [Info] 		Index total memory:13621658091 (12.6862G), mem:10444486052 (9.72719G), overhead:3177172039 (2.95897G), min:10444486052 (9.72719G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.592+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.592+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.592+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.592+05:30 [Info] 		Index total memory:13855332907 (12.9038G), mem:10432434095 (9.71596G), overhead:3422898812 (3.18782G), min:10432434095 (9.71596G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.592+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.592+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.592+05:30 [Info] 		Index name:idx1 0, bucket:bucket2, scope:, collection:, defnId:987654, instId:11972398092819331361, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.592+05:30 [Info] 		Index total memory:7299864463 (6.79853G), mem:7299864463 (6.79853G), overhead:0 (0), min:7299864463 (6.79853G), data:7299864463 (6.79853G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.592+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.592+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.592+05:30 [Info] 		Index name:idx1 4, bucket:bucket2, scope:, collection:, defnId:987654, instId:11972398092819331361, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.592+05:30 [Info] 		Index total memory:7299864463 (6.79853G), mem:7299864463 (6.79853G), overhead:0 (0), min:7299864463 (6.79853G), data:7299864463 (6.79853G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.592+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.593+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.593+05:30 [Info] 		Index name:idx1 6, bucket:bucket2, scope:, collection:, defnId:987654, instId:11972398092819331361, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.593+05:30 [Info] 		Index total memory:7299864463 (6.79853G), mem:7299864463 (6.79853G), overhead:0 (0), min:7299864463 (6.79853G), data:7299864463 (6.79853G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.593+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.593+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.593+05:30 [Info] 		Index name:idx1 2 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:11972398092819331361, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.593+05:30 [Info] 		Index total memory:7299864463 (6.79853G), mem:7299864463 (6.79853G), overhead:0 (0), min:7299864463 (6.79853G), data:7299864463 (6.79853G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.593+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.593+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.593+05:30 [Info] 		Index name:idx1 3, bucket:bucket2, scope:, collection:, defnId:987654, instId:11972398092819331361, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.593+05:30 [Info] 		Index total memory:7299864463 (6.79853G), mem:7299864463 (6.79853G), overhead:0 (0), min:7299864463 (6.79853G), data:7299864463 (6.79853G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.593+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.593+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.593+05:30 [Info] 		Index name:idx1 1, bucket:bucket2, scope:, collection:, defnId:987654, instId:11972398092819331361, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.593+05:30 [Info] 		Index total memory:7299864463 (6.79853G), mem:7299864463 (6.79853G), overhead:0 (0), min:7299864463 (6.79853G), data:7299864463 (6.79853G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.593+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023/09/01 13:38:27 -------------------------------------------
2023/09/01 13:38:27 Place Paritioned Index with 2 replica - 3 non-empty heterogenous nodes with 1 Scaled up
2023-09-01T13:38:27.595+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.23935882118177618
2023-09-01T13:38:27.635+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:27.635+05:30 [Info] Score: 0.14532097559770876
2023-09-01T13:38:27.635+05:30 [Info] variation: 0.2906419511954175
2023-09-01T13:38:27.635+05:30 [Info] ElapsedTime: 39ms
2023-09-01T13:38:27.635+05:30 [Info] ConvergenceTime: 6ms
2023-09-01T13:38:27.635+05:30 [Info] Iteration: 7
2023-09-01T13:38:27.635+05:30 [Info] Move: 7
2023-09-01T13:38:27.635+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.635+05:30 [Info] Using SAPlanner
2023-09-01T13:38:27.635+05:30 [Info] Indexer Memory Mean 156291089214 (145.557G)
2023-09-01T13:38:27.635+05:30 [Info] Indexer Memory Deviation 90849494247 (84.6102G) (58.13%)
2023-09-01T13:38:27.635+05:30 [Info] Indexer Memory Utilization 2.8335
2023-09-01T13:38:27.635+05:30 [Info] Indexer CPU Mean 15.3333
2023-09-01T13:38:27.635+05:30 [Info] Indexer CPU Deviation 1.70 (11.08%)
2023-09-01T13:38:27.635+05:30 [Info] Indexer CPU Utilization 0.5679
2023-09-01T13:38:27.635+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:27.635+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:27.635+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:27.635+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:27.635+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:27.635+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:27.635+05:30 [Info] Indexer Data Size Mean 47484620728 (44.2235G)
2023-09-01T13:38:27.635+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2023-09-01T13:38:27.635+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:27.635+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:27.635+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:27.635+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:27.635+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.635+05:30 [Info] Number of indexes: 36
2023-09-01T13:38:27.635+05:30 [Info] Number of indexers: 3
2023-09-01T13:38:27.635+05:30 [Info] Avg Index Size: 13024257434 (12.1298G)
2023-09-01T13:38:27.635+05:30 [Info] Max Index Size: 73059719100 (68.0422G)
2023-09-01T13:38:27.635+05:30 [Info] Max Indexer Overhead: 30560384804 (28.4616G)
2023-09-01T13:38:27.635+05:30 [Info] Avg Index Cpu: 1.2778
2023-09-01T13:38:27.635+05:30 [Info] Max Index Cpu: 5.0000
2023-09-01T13:38:27.635+05:30 [Info] Num Estimation Run: 32
2023-09-01T13:38:27.635+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.635+05:30 [Info] Memory Quota 55158315092 (51.3702G)
2023-09-01T13:38:27.635+05:30 [Info] CPU Quota 27
2023-09-01T13:38:27.635+05:30 [Info] Max Cpu Utilization -1
2023-09-01T13:38:27.635+05:30 [Info] Max Memory Utilization -1
2023-09-01T13:38:27.635+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.635+05:30 [Info] Total iteration: 6744
2023-09-01T13:38:27.635+05:30 [Info] RandomSwap time: 0ns
2023-09-01T13:38:27.635+05:30 [Info] RandomSwap call: 0
2023-09-01T13:38:27.635+05:30 [Info] RandomSwap iteration: 0
2023-09-01T13:38:27.635+05:30 [Info] RandomMove time: 55us
2023-09-01T13:38:27.635+05:30 [Info] RandomMove call: 6 (empty index 0)
2023-09-01T13:38:27.635+05:30 [Info] RandomMove average time: 9us
2023-09-01T13:38:27.635+05:30 [Info] ExhaustMove time: 16us
2023-09-01T13:38:27.635+05:30 [Info] ExhaustMove call: 1
2023-09-01T13:38:27.635+05:30 [Info] ExhaustMove average time: 16us
2023-09-01T13:38:27.635+05:30 [Info] ExhaustSwap time: 0ns
2023-09-01T13:38:27.635+05:30 [Info] ExhaustSwap call: 0
2023-09-01T13:38:27.635+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.635+05:30 [Info] 
2023-09-01T13:38:27.635+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9000, nodeUUID:tempNodeUUID_2596996162, useLiveData:true
2023-09-01T13:38:27.635+05:30 [Info] Indexer total memory:315266309837 (293.615G), mem:284705925033 (265.153G), overhead:30560384804 (28.4616G), min:47484620728 (44.2235G), data:47484620728 (44.2235G) cpu:17.0000, io:0 (0), scan:0 drain:0 numIndexes:12
2023-09-01T13:38:27.635+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.5643277764120702 allowDDLDuringScaleup:true
2023-09-01T13:38:27.636+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.636+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.636+05:30 [Info] 		Index total memory:30961528072 (28.8352G), mem:27754610480 (25.8485G), overhead:3206917592 (2.98667G), min:5550922096 (5.1697G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.636+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.636+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.636+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.636+05:30 [Info] 		Index total memory:81043548208 (75.4777G), mem:73059719100 (68.0422G), overhead:7983829108 (7.43552G), min:14611943820 (13.6084G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.636+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.636+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.636+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.636+05:30 [Info] 		Index total memory:82702164802 (77.0224G), mem:72507837150 (67.5282G), overhead:10194327652 (9.49421G), min:14501567430 (13.5056G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.636+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.636+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.636+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.636+05:30 [Info] 		Index total memory:73074448027 (68.0559G), mem:63899137575 (59.5107G), overhead:9175310452 (8.54517G), min:12779827515 (11.9021G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.636+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.636+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.636+05:30 [Info] 		Index name:idx1 1, bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.636+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.636+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.636+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.636+05:30 [Info] 		Index name:idx1 6, bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.636+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.636+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.636+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.636+05:30 [Info] 		Index name:idx1 4 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.636+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.636+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.636+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.636+05:30 [Info] 		Index name:idx1 5 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.636+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.636+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.636+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.636+05:30 [Info] 		Index name:idx1 3 (replica 2), bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.636+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.636+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.636+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.636+05:30 [Info] 		Index name:idx1 2, bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.636+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.636+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.636+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.636+05:30 [Info] 		Index name:idx1 0 (replica 2), bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.636+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.636+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.636+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.636+05:30 [Info] 		Index name:idx1 7 (replica 2), bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.636+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.636+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.636+05:30 [Info] 
2023-09-01T13:38:27.636+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9001, nodeUUID:tempNodeUUID_4039455774, useLiveData:true
2023-09-01T13:38:27.636+05:30 [Info] Indexer total memory:104855561729 (97.6544G), mem:88524406791 (82.4448G), overhead:16331154938 (15.2096G), min:47484620728 (44.2235G), data:47484620728 (44.2235G) cpu:13.0000, io:0 (0), scan:0 drain:0 numIndexes:12
2023-09-01T13:38:27.636+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.3132814509590003 allowDDLDuringScaleup:true
2023-09-01T13:38:27.636+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.636+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.636+05:30 [Info] 		Index total memory:8339546089 (7.76681G), mem:5550922096 (5.1697G), overhead:2788623993 (2.59711G), min:5550922096 (5.1697G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.636+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.636+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.636+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.636+05:30 [Info] 		Index total memory:21554403914 (20.0741G), mem:14611943820 (13.6084G), overhead:6942460094 (6.46567G), min:14611943820 (13.6084G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.636+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.636+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.636+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.636+05:30 [Info] 		Index total memory:13621658091 (12.6862G), mem:10444486052 (9.72719G), overhead:3177172039 (2.95897G), min:10444486052 (9.72719G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.636+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.636+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.636+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.636+05:30 [Info] 		Index total memory:13855332907 (12.9038G), mem:10432434095 (9.71596G), overhead:3422898812 (3.18782G), min:10432434095 (9.71596G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.636+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.636+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.636+05:30 [Info] 		Index name:idx1 3, bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.636+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.636+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.636+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.636+05:30 [Info] 		Index name:idx1 0 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.636+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.637+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.637+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.637+05:30 [Info] 		Index name:idx1 7 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.637+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.637+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.637+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.637+05:30 [Info] 		Index name:idx1 1 (replica 2), bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.637+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.637+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.637+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.637+05:30 [Info] 		Index name:idx1 2 (replica 2), bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.637+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.637+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.637+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.637+05:30 [Info] 		Index name:idx1 5 (replica 2), bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.637+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.637+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.637+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.637+05:30 [Info] 		Index name:idx1 6 (replica 2), bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.637+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.637+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.637+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.637+05:30 [Info] 		Index name:idx1 4 (replica 2), bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.637+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.637+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.637+05:30 [Info] 
2023-09-01T13:38:27.637+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9002, nodeUUID:tempNodeUUID_2854263694, useLiveData:true
2023-09-01T13:38:27.637+05:30 [Info] Indexer total memory:119086170240 (110.908G), mem:95642935820 (89.0744G), overhead:23443234420 (21.8332G), min:47484620728 (44.2235G), data:47484620728 (44.2235G) cpu:16.0000, io:0 (0), scan:0 drain:0 numIndexes:12
2023-09-01T13:38:27.637+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.3223907726289295 allowDDLDuringScaleup:true
2023-09-01T13:38:27.637+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.637+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.637+05:30 [Info] 		Index total memory:23366200171 (21.7615G), mem:14501567430 (13.5056G), overhead:8864632741 (8.25583G), min:14501567430 (13.5056G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.637+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.637+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.637+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.637+05:30 [Info] 		Index total memory:20758358343 (19.3327G), mem:12779827515 (11.9021G), overhead:7978530828 (7.43059G), min:12779827515 (11.9021G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.637+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.637+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.637+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.637+05:30 [Info] 		Index total memory:13621658091 (12.6862G), mem:10444486052 (9.72719G), overhead:3177172039 (2.95897G), min:10444486052 (9.72719G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.637+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.637+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.637+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.637+05:30 [Info] 		Index total memory:13855332907 (12.9038G), mem:10432434095 (9.71596G), overhead:3422898812 (3.18782G), min:10432434095 (9.71596G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.637+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.637+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.637+05:30 [Info] 		Index name:idx1 0, bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.637+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.637+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.637+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.637+05:30 [Info] 		Index name:idx1 4, bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.637+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.637+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.637+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.637+05:30 [Info] 		Index name:idx1 1 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.637+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.637+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.637+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.637+05:30 [Info] 		Index name:idx1 2 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.637+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.637+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.637+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.637+05:30 [Info] 		Index name:idx1 3 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.637+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.637+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.637+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.637+05:30 [Info] 		Index name:idx1 7, bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.637+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.637+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.637+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.637+05:30 [Info] 		Index name:idx1 6 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.638+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.638+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.638+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.638+05:30 [Info] 		Index name:idx1 5, bucket:bucket2, scope:, collection:, defnId:987654, instId:2393892648593517155, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.638+05:30 [Info] 		Index total memory:5935577591 (5.52794G), mem:5935577591 (5.52794G), overhead:0 (0), min:5935577591 (5.52794G), data:5935577591 (5.52794G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.638+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023/09/01 13:38:27 -------------------------------------------
2023/09/01 13:38:27 Place Paritioned Index with 2 replica - 3 non-empty heterogenous nodes with 2 Scaled up
2023-09-01T13:38:27.640+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.24558624580415575
2023-09-01T13:38:27.675+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:27.675+05:30 [Info] Score: 0.09796819540510597
2023-09-01T13:38:27.675+05:30 [Info] variation: 0.19593639081021194
2023-09-01T13:38:27.675+05:30 [Info] ElapsedTime: 34ms
2023-09-01T13:38:27.675+05:30 [Info] ConvergenceTime: 5ms
2023-09-01T13:38:27.675+05:30 [Info] Iteration: 7
2023-09-01T13:38:27.675+05:30 [Info] Move: 7
2023-09-01T13:38:27.675+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.675+05:30 [Info] Using SAPlanner
2023-09-01T13:38:27.675+05:30 [Info] Indexer Memory Mean 210829304949 (196.35G)
2023-09-01T13:38:27.675+05:30 [Info] Indexer Memory Deviation 82618266177 (76.9443G) (39.19%)
2023-09-01T13:38:27.675+05:30 [Info] Indexer Memory Utilization 3.8223
2023-09-01T13:38:27.675+05:30 [Info] Indexer CPU Mean 15.3333
2023-09-01T13:38:27.675+05:30 [Info] Indexer CPU Deviation 1.70 (11.08%)
2023-09-01T13:38:27.675+05:30 [Info] Indexer CPU Utilization 0.5679
2023-09-01T13:38:27.675+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:27.675+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:27.675+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:27.675+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:27.675+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:27.675+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:27.675+05:30 [Info] Indexer Data Size Mean 47303121712 (44.0545G)
2023-09-01T13:38:27.675+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2023-09-01T13:38:27.675+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:27.675+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:27.675+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:27.675+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:27.675+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.675+05:30 [Info] Number of indexes: 36
2023-09-01T13:38:27.675+05:30 [Info] Number of indexers: 3
2023-09-01T13:38:27.675+05:30 [Info] Avg Index Size: 17569108745 (16.3625G)
2023-09-01T13:38:27.675+05:30 [Info] Max Index Size: 73059719100 (68.0422G)
2023-09-01T13:38:27.675+05:30 [Info] Max Indexer Overhead: 81655774690 (76.0479G)
2023-09-01T13:38:27.675+05:30 [Info] Avg Index Cpu: 1.2778
2023-09-01T13:38:27.675+05:30 [Info] Max Index Cpu: 5.0000
2023-09-01T13:38:27.675+05:30 [Info] Num Estimation Run: 33
2023-09-01T13:38:27.675+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.675+05:30 [Info] Memory Quota 55158315092 (51.3702G)
2023-09-01T13:38:27.675+05:30 [Info] CPU Quota 27
2023-09-01T13:38:27.675+05:30 [Info] Max Cpu Utilization -1
2023-09-01T13:38:27.675+05:30 [Info] Max Memory Utilization -1
2023-09-01T13:38:27.675+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.675+05:30 [Info] Total iteration: 6761
2023-09-01T13:38:27.675+05:30 [Info] RandomSwap time: 0ns
2023-09-01T13:38:27.675+05:30 [Info] RandomSwap call: 0
2023-09-01T13:38:27.675+05:30 [Info] RandomSwap iteration: 0
2023-09-01T13:38:27.675+05:30 [Info] RandomMove time: 55us
2023-09-01T13:38:27.675+05:30 [Info] RandomMove call: 6 (empty index 0)
2023-09-01T13:38:27.675+05:30 [Info] RandomMove average time: 9us
2023-09-01T13:38:27.675+05:30 [Info] ExhaustMove time: 11us
2023-09-01T13:38:27.675+05:30 [Info] ExhaustMove call: 1
2023-09-01T13:38:27.675+05:30 [Info] ExhaustMove average time: 11us
2023-09-01T13:38:27.675+05:30 [Info] ExhaustSwap time: 0ns
2023-09-01T13:38:27.675+05:30 [Info] ExhaustSwap call: 0
2023-09-01T13:38:27.675+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.675+05:30 [Info] 
2023-09-01T13:38:27.675+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9000, nodeUUID:tempNodeUUID_2596996162, useLiveData:true
2023-09-01T13:38:27.675+05:30 [Info] Indexer total memory:315084810821 (293.446G), mem:284524426017 (264.984G), overhead:30560384804 (28.4616G), min:47303121712 (44.0545G), data:47303121712 (44.0545G) cpu:17.0000, io:0 (0), scan:0 drain:0 numIndexes:12
2023-09-01T13:38:27.675+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.4699097510048492 allowDDLDuringScaleup:true
2023-09-01T13:38:27.675+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.675+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.676+05:30 [Info] 		Index total memory:30961528072 (28.8352G), mem:27754610480 (25.8485G), overhead:3206917592 (2.98667G), min:5550922096 (5.1697G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.676+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.676+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.676+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.676+05:30 [Info] 		Index total memory:81043548208 (75.4777G), mem:73059719100 (68.0422G), overhead:7983829108 (7.43552G), min:14611943820 (13.6084G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.676+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.676+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.676+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.676+05:30 [Info] 		Index total memory:82702164802 (77.0224G), mem:72507837150 (67.5282G), overhead:10194327652 (9.49421G), min:14501567430 (13.5056G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.676+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.676+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.676+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:0, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.676+05:30 [Info] 		Index total memory:73074448027 (68.0559G), mem:63899137575 (59.5107G), overhead:9175310452 (8.54517G), min:12779827515 (11.9021G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.676+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.676+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.676+05:30 [Info] 		Index name:idx1 3, bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.676+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.676+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.676+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.676+05:30 [Info] 		Index name:idx1 0 (replica 2), bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.676+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.676+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.676+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.676+05:30 [Info] 		Index name:idx1 4 (replica 2), bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.676+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.676+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.676+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.676+05:30 [Info] 		Index name:idx1 6 (replica 2), bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.676+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.676+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.676+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.676+05:30 [Info] 		Index name:idx1 2 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.676+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.676+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.676+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.676+05:30 [Info] 		Index name:idx1 1 (replica 2), bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.676+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.676+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.676+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.676+05:30 [Info] 		Index name:idx1 5 (replica 2), bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.676+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.676+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.676+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.676+05:30 [Info] 		Index name:idx1 7 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.676+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.676+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.676+05:30 [Info] 
2023-09-01T13:38:27.676+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9001, nodeUUID:tempNodeUUID_4039455774, useLiveData:true
2023-09-01T13:38:27.676+05:30 [Info] Indexer total memory:334157826717 (311.209G), mem:252502052027 (235.161G), overhead:81655774690 (76.0479G), min:47303121712 (44.0545G), data:47303121712 (44.0545G) cpu:13.0000, io:0 (0), scan:0 drain:0 numIndexes:12
2023-09-01T13:38:27.676+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.43953221501885753 allowDDLDuringScaleup:true
2023-09-01T13:38:27.676+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.676+05:30 [Info] 		Index name:1, bucket:bucket1, scope:, collection:, defnId:1, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.676+05:30 [Info] 		Index total memory:30961528072 (28.8352G), mem:27754610480 (25.8485G), overhead:3206917592 (2.98667G), min:5550922096 (5.1697G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.676+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.676+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.676+05:30 [Info] 		Index name:2, bucket:bucket1, scope:, collection:, defnId:2, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.676+05:30 [Info] 		Index total memory:81043548208 (75.4777G), mem:73059719100 (68.0422G), overhead:7983829108 (7.43552G), min:14611943820 (13.6084G), data:0 (0) cpu:4.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.676+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.676+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.676+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.676+05:30 [Info] 		Index total memory:55876178105 (52.0387G), mem:52222430260 (48.6359G), overhead:3653747845 (3.40282G), min:10444486052 (9.72719G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.676+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.676+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.676+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.676+05:30 [Info] 		Index total memory:56098504109 (52.2458G), mem:52162170475 (48.5798G), overhead:3936333634 (3.666G), min:10432434095 (9.71596G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.676+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.676+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.676+05:30 [Info] 		Index name:idx1 4, bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.676+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.676+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.676+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.676+05:30 [Info] 		Index name:idx1 5, bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.676+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.677+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.677+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.677+05:30 [Info] 		Index name:idx1 6, bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.677+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.677+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.677+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.677+05:30 [Info] 		Index name:idx1 1 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.677+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.677+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.677+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.677+05:30 [Info] 		Index name:idx1 2 (replica 2), bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.677+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.677+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.677+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.677+05:30 [Info] 		Index name:idx1 3 (replica 2), bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.677+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.677+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.677+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.677+05:30 [Info] 		Index name:idx1 7 (replica 2), bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.677+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.677+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.677+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.677+05:30 [Info] 		Index name:idx1 0 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.677+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.677+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.677+05:30 [Info] 
2023-09-01T13:38:27.677+05:30 [Info] Indexer serverGroup:, nodeId:127.0.0.1:9002, nodeUUID:tempNodeUUID_2854263694, useLiveData:true
2023-09-01T13:38:27.677+05:30 [Info] Indexer total memory:118904671224 (110.739G), mem:95461436804 (88.9054G), overhead:23443234420 (21.8332G), min:47303121712 (44.0545G), data:47303121712 (44.0545G) cpu:16.0000, io:0 (0), scan:0 drain:0 numIndexes:12
2023-09-01T13:38:27.677+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.29055803397629315 allowDDLDuringScaleup:true
2023-09-01T13:38:27.677+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.677+05:30 [Info] 		Index name:3, bucket:bucket1, scope:, collection:, defnId:3, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.677+05:30 [Info] 		Index total memory:23366200171 (21.7615G), mem:14501567430 (13.5056G), overhead:8864632741 (8.25583G), min:14501567430 (13.5056G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.677+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.677+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.677+05:30 [Info] 		Index name:4, bucket:bucket1, scope:, collection:, defnId:4, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.677+05:30 [Info] 		Index total memory:20758358343 (19.3327G), mem:12779827515 (11.9021G), overhead:7978530828 (7.43059G), min:12779827515 (11.9021G), data:0 (0) cpu:5.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.677+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.677+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.677+05:30 [Info] 		Index name:5, bucket:bucket1, scope:, collection:, defnId:5, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.677+05:30 [Info] 		Index total memory:13621658091 (12.6862G), mem:10444486052 (9.72719G), overhead:3177172039 (2.95897G), min:10444486052 (9.72719G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.677+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.677+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.677+05:30 [Info] 		Index name:6, bucket:bucket1, scope:, collection:, defnId:6, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.677+05:30 [Info] 		Index total memory:13855332907 (12.9038G), mem:10432434095 (9.71596G), overhead:3422898812 (3.18782G), min:10432434095 (9.71596G), data:0 (0) cpu:3.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.677+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.677+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.677+05:30 [Info] 		Index name:idx1 0, bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.677+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.677+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.677+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.677+05:30 [Info] 		Index name:idx1 1, bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.677+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.677+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.677+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.677+05:30 [Info] 		Index name:idx1 2, bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.677+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.677+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.677+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.677+05:30 [Info] 		Index name:idx1 7, bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.677+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.677+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.677+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.677+05:30 [Info] 		Index name:idx1 4 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.677+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.677+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.677+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.677+05:30 [Info] 		Index name:idx1 5 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.677+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.677+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.677+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.677+05:30 [Info] 		Index name:idx1 3 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.677+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.677+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.677+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.677+05:30 [Info] 		Index name:idx1 6 (replica 1), bucket:bucket2, scope:, collection:, defnId:987654, instId:14133188833886684314, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.677+05:30 [Info] 		Index total memory:5912890214 (5.50681G), mem:5912890214 (5.50681G), overhead:0 (0), min:5912890214 (5.50681G), data:5912890214 (5.50681G) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.677+05:30 [Info] 		Index resident:0% build:0% estimated:true equivCheck:true pendingCreate:false pendingDelete:false
=== RUN   TestPlanDuringHeterogenousScaleup/CommandRepair
2023/09/01 13:38:27 -------------------------------------------
2023/09/01 13:38:27 Increment Replica count by 1 - 3 non-empty heterogenous nodes with 1 Scaled up
2023-09-01T13:38:27.678+05:30 [Info] Rebuilding lost replica for (default,,,replicaIdx2,2,0) with initial placement on node: 127.0.0.1:9001
2023-09-01T13:38:27.678+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=1.  Elapsed Time=5us
2023-09-01T13:38:27.678+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=2.  Elapsed Time=6us
2023-09-01T13:38:27.678+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=3.  Elapsed Time=6us
2023-09-01T13:38:27.678+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=4.  Elapsed Time=5us
2023-09-01T13:38:27.679+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.8995910950151497
2023-09-01T13:38:27.679+05:30 [Info] Planner::initial temperature: initial resource variation 0.8995910950151497 temp 0.08995910950151498
2023-09-01T13:38:27.679+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (0, 0) exceeded.
2023-09-01T13:38:27.679+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:27.679+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=5.  Elapsed Time=130us
2023-09-01T13:38:27.679+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.8995910950151497
2023-09-01T13:38:27.679+05:30 [Info] Planner::initial temperature: initial resource variation 0.8995910950151497 temp 0.08995910950151498
2023-09-01T13:38:27.679+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (0, 0) exceeded.
2023-09-01T13:38:27.679+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:27.679+05:30 [Info] Score: 0.22489777375378742
2023-09-01T13:38:27.679+05:30 [Info] variation: 0.8995910950151497
2023-09-01T13:38:27.679+05:30 [Info] ElapsedTime: 72us
2023-09-01T13:38:27.679+05:30 [Info] ConvergenceTime: 2us
2023-09-01T13:38:27.679+05:30 [Info] Iteration: 0
2023-09-01T13:38:27.679+05:30 [Info] Move: 0
2023-09-01T13:38:27.679+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.679+05:30 [Info] Using SAPlanner
2023-09-01T13:38:27.679+05:30 [Info] Indexer Memory Mean 122486397174 (114.074G)
2023-09-01T13:38:27.679+05:30 [Info] Indexer Memory Deviation 110187672158 (102.62G) (89.96%)
2023-09-01T13:38:27.679+05:30 [Info] Indexer Memory Utilization 2.2206
2023-09-01T13:38:27.679+05:30 [Info] Indexer CPU Mean 0.0000
2023-09-01T13:38:27.679+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2023-09-01T13:38:27.679+05:30 [Info] Indexer CPU Utilization 0.0000
2023-09-01T13:38:27.679+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:27.679+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:27.679+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:27.679+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:27.679+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:27.679+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:27.679+05:30 [Info] Indexer Data Size Mean 0 (0)
2023-09-01T13:38:27.679+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2023-09-01T13:38:27.679+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:27.679+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:27.679+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:27.679+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:27.679+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.679+05:30 [Info] Number of indexes: 4
2023-09-01T13:38:27.679+05:30 [Info] Number of indexers: 3
2023-09-01T13:38:27.679+05:30 [Info] Avg Index Size: 91864797880 (85.5558G)
2023-09-01T13:38:27.679+05:30 [Info] Max Index Size: 237221304305 (220.93G)
2023-09-01T13:38:27.679+05:30 [Info] Max Indexer Overhead: 46891539742 (43.6711G)
2023-09-01T13:38:27.679+05:30 [Info] Avg Index Cpu: 0.0000
2023-09-01T13:38:27.679+05:30 [Info] Max Index Cpu: 0.0000
2023-09-01T13:38:27.679+05:30 [Info] Num Estimation Run: 0
2023-09-01T13:38:27.679+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.679+05:30 [Info] Memory Quota 55158315092 (51.3702G)
2023-09-01T13:38:27.679+05:30 [Info] CPU Quota 27
2023-09-01T13:38:27.679+05:30 [Info] Max Cpu Utilization -1
2023-09-01T13:38:27.679+05:30 [Info] Max Memory Utilization -1
2023-09-01T13:38:27.679+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.679+05:30 [Info] Total iteration: 0
2023-09-01T13:38:27.679+05:30 [Info] RandomSwap time: 0ns
2023-09-01T13:38:27.679+05:30 [Info] RandomSwap call: 0
2023-09-01T13:38:27.679+05:30 [Info] RandomSwap iteration: 0
2023-09-01T13:38:27.679+05:30 [Info] RandomMove time: 0ns
2023-09-01T13:38:27.679+05:30 [Info] RandomMove call: 0 (empty index 0)
2023-09-01T13:38:27.679+05:30 [Info] ExhaustMove time: 0ns
2023-09-01T13:38:27.679+05:30 [Info] ExhaustMove call: 0
2023-09-01T13:38:27.679+05:30 [Info] ExhaustSwap time: 0ns
2023-09-01T13:38:27.679+05:30 [Info] ExhaustSwap call: 0
2023-09-01T13:38:27.679+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.679+05:30 [Info] 
2023-09-01T13:38:27.679+05:30 [Info] Indexer serverGroup:sg0, nodeId:127.0.0.1:9001, nodeUUID:af33bb1cf058349f6fafffa6a843a979, useLiveData:true
2023-09-01T13:38:27.679+05:30 [Info] Indexer total memory:325152630110 (302.822G), mem:278261090368 (259.151G), overhead:46891539742 (43.6711G), min:52248944865 (48.6606G), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:2
2023-09-01T13:38:27.679+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.45435427408637796 allowDDLDuringScaleup:true
2023-09-01T13:38:27.679+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.679+05:30 [Info] 		Index name:replicaIdx1 0, bucket:default, scope:, collection:, defnId:1337, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.679+05:30 [Info] 		Index total memory:267781689109 (249.391G), mem:237221304305 (220.93G), overhead:30560384804 (28.4616G), min:28015361594 (26.0913G), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.679+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.679+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.679+05:30 [Info] 		Index name:replicaIdx2 0 (replica 2), bucket:default, scope:, collection:, defnId:4242, instId:3131078705869069389, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.679+05:30 [Info] 		Index total memory:57370941001 (53.4309G), mem:41039786063 (38.2213G), overhead:16331154938 (15.2096G), min:24233583271 (22.5693G), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.679+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.679+05:30 [Info] 
2023-09-01T13:38:27.679+05:30 [Info] Indexer serverGroup:sg0, nodeId:127.0.0.1:9002, nodeUUID:929bb73f8aaeb55422bdc071cf2dfbfc, useLiveData:true
2023-09-01T13:38:27.679+05:30 [Info] Indexer total memory:57370941001 (53.4309G), mem:41039786063 (38.2213G), overhead:16331154938 (15.2096G), min:24233583271 (22.5693G), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2023-09-01T13:38:27.679+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.06701117350131312 allowDDLDuringScaleup:true
2023-09-01T13:38:27.679+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.680+05:30 [Info] 		Index name:replicaIdx2 0 (replica 1), bucket:default, scope:, collection:, defnId:4242, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.680+05:30 [Info] 		Index total memory:57370941001 (53.4309G), mem:41039786063 (38.2213G), overhead:16331154938 (15.2096G), min:24233583271 (22.5693G), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.680+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.680+05:30 [Info] 
2023-09-01T13:38:27.680+05:30 [Info] Indexer serverGroup:sg0, nodeId:127.0.0.1:9003, nodeUUID:934d23a204d25a4970e0262fa63920ba, useLiveData:true
2023-09-01T13:38:27.680+05:30 [Info] Indexer total memory:71601549512 (66.6841G), mem:48158315092 (44.8509G), overhead:23443234420 (21.8332G), min:28437003477 (26.484G), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2023-09-01T13:38:27.680+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0.0786345524123089 allowDDLDuringScaleup:true
2023-09-01T13:38:27.680+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.680+05:30 [Info] 		Index name:replicaIdx2 0, bucket:default, scope:, collection:, defnId:4242, instId:2, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.680+05:30 [Info] 		Index total memory:71601549512 (66.6841G), mem:48158315092 (44.8509G), overhead:23443234420 (21.8332G), min:28437003477 (26.484G), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.680+05:30 [Info] 		Index resident:0% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023/09/01 13:38:27 -------------------------------------------
2023/09/01 13:38:27 Increment Replica count by 2 - 3 non-empty heterogenous nodes with 1 Scaled up
2023-09-01T13:38:27.680+05:30 [Info] Rebuilding lost replica for (default,,,replicaIdx1,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:27.680+05:30 [Info] Rebuilding lost replica for (default,,,replicaIdx1,2,0) with initial placement on node: 127.0.0.1:9002
2023-09-01T13:38:27.680+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=1.  Elapsed Time=4us
2023-09-01T13:38:27.680+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=2.  Elapsed Time=4us
2023-09-01T13:38:27.680+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=3.  Elapsed Time=3us
2023-09-01T13:38:27.680+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=4.  Elapsed Time=6us
2023-09-01T13:38:27.680+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=5.  Elapsed Time=7us
2023-09-01T13:38:27.680+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.0795046976786775
2023-09-01T13:38:27.680+05:30 [Info] Planner::initial temperature: initial resource variation 0.0795046976786775 temp 0.00795046976786775
2023-09-01T13:38:27.680+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (0, 0) exceeded.
2023-09-01T13:38:27.680+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:27.680+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=6.  Elapsed Time=126us
2023-09-01T13:38:27.680+05:30 [Warn] Unable to find a solution with resource constraint.  Relax resource constraint check.
2023-09-01T13:38:27.680+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.0795046976786775
2023-09-01T13:38:27.680+05:30 [Info] Planner::initial temperature: initial resource variation 0.0795046976786775 temp 0.00795046976786775
2023-09-01T13:38:27.680+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (0, 0) exceeded.
2023-09-01T13:38:27.680+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:27.680+05:30 [Info] Score: 0.019876174419669376
2023-09-01T13:38:27.680+05:30 [Info] variation: 0.0795046976786775
2023-09-01T13:38:27.680+05:30 [Info] ElapsedTime: 74us
2023-09-01T13:38:27.681+05:30 [Info] ConvergenceTime: 2us
2023-09-01T13:38:27.681+05:30 [Info] Iteration: 0
2023-09-01T13:38:27.681+05:30 [Info] Move: 0
2023-09-01T13:38:27.681+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.681+05:30 [Info] Using SAPlanner
2023-09-01T13:38:27.681+05:30 [Info] Indexer Memory Mean 266954004690 (248.62G)
2023-09-01T13:38:27.681+05:30 [Info] Indexer Memory Deviation 21224097436 (19.7665G) (7.95%)
2023-09-01T13:38:27.681+05:30 [Info] Indexer Memory Utilization 4.8398
2023-09-01T13:38:27.681+05:30 [Info] Indexer CPU Mean 0.0000
2023-09-01T13:38:27.681+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2023-09-01T13:38:27.681+05:30 [Info] Indexer CPU Utilization 0.0000
2023-09-01T13:38:27.681+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:27.681+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:27.681+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:27.681+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:27.681+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:27.681+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:27.681+05:30 [Info] Indexer Data Size Mean 0 (0)
2023-09-01T13:38:27.681+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2023-09-01T13:38:27.681+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:27.681+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:27.681+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:27.681+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:27.681+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.681+05:30 [Info] Number of indexes: 5
2023-09-01T13:38:27.681+05:30 [Info] Number of indexers: 3
2023-09-01T13:38:27.681+05:30 [Info] Avg Index Size: 160172402814 (149.172G)
2023-09-01T13:38:27.681+05:30 [Info] Max Index Size: 237221304305 (220.93G)
2023-09-01T13:38:27.681+05:30 [Info] Max Indexer Overhead: 54003619224 (50.2948G)
2023-09-01T13:38:27.681+05:30 [Info] Avg Index Cpu: 0.0000
2023-09-01T13:38:27.681+05:30 [Info] Max Index Cpu: 0.0000
2023-09-01T13:38:27.681+05:30 [Info] Num Estimation Run: 0
2023-09-01T13:38:27.681+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.681+05:30 [Info] Memory Quota 55158315092 (51.3702G)
2023-09-01T13:38:27.681+05:30 [Info] CPU Quota 27
2023-09-01T13:38:27.681+05:30 [Info] Max Cpu Utilization -1
2023-09-01T13:38:27.681+05:30 [Info] Max Memory Utilization -1
2023-09-01T13:38:27.681+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.681+05:30 [Info] Total iteration: 0
2023-09-01T13:38:27.681+05:30 [Info] RandomSwap time: 0ns
2023-09-01T13:38:27.681+05:30 [Info] RandomSwap call: 0
2023-09-01T13:38:27.681+05:30 [Info] RandomSwap iteration: 0
2023-09-01T13:38:27.681+05:30 [Info] RandomMove time: 0ns
2023-09-01T13:38:27.681+05:30 [Info] RandomMove call: 0 (empty index 0)
2023-09-01T13:38:27.681+05:30 [Info] ExhaustMove time: 0ns
2023-09-01T13:38:27.681+05:30 [Info] ExhaustMove call: 0
2023-09-01T13:38:27.681+05:30 [Info] ExhaustSwap time: 0ns
2023-09-01T13:38:27.681+05:30 [Info] ExhaustSwap call: 0
2023-09-01T13:38:27.681+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.681+05:30 [Info] 
2023-09-01T13:38:27.681+05:30 [Info] Indexer serverGroup:sg0, nodeId:127.0.0.1:9001, nodeUUID:af33bb1cf058349f6fafffa6a843a979, useLiveData:true
2023-09-01T13:38:27.681+05:30 [Info] Indexer total memory:267781689109 (249.391G), mem:237221304305 (220.93G), overhead:30560384804 (28.4616G), min:25213825434 (23.4822G), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2023-09-01T13:38:27.681+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.1777244769790758 allowDDLDuringScaleup:true
2023-09-01T13:38:27.681+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.681+05:30 [Info] 		Index name:replicaIdx1 0, bucket:default, scope:, collection:, defnId:1337, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.681+05:30 [Info] 		Index total memory:267781689109 (249.391G), mem:237221304305 (220.93G), overhead:30560384804 (28.4616G), min:25213825434 (23.4822G), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.681+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.681+05:30 [Info] 
2023-09-01T13:38:27.681+05:30 [Info] Indexer serverGroup:sg0, nodeId:127.0.0.1:9002, nodeUUID:929bb73f8aaeb55422bdc071cf2dfbfc, useLiveData:true
2023-09-01T13:38:27.681+05:30 [Info] Indexer total memory:325152630110 (302.822G), mem:278261090368 (259.151G), overhead:46891539742 (43.6711G), min:47024050377 (43.7946G), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:2
2023-09-01T13:38:27.681+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.20847118640616036 allowDDLDuringScaleup:true
2023-09-01T13:38:27.681+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.681+05:30 [Info] 		Index name:replicaIdx2 0 (replica 1), bucket:default, scope:, collection:, defnId:4242, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.681+05:30 [Info] 		Index total memory:57370941001 (53.4309G), mem:41039786063 (38.2213G), overhead:16331154938 (15.2096G), min:21810224943 (20.3124G), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.681+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.681+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.681+05:30 [Info] 		Index name:replicaIdx1 0 (replica 2), bucket:default, scope:, collection:, defnId:1337, instId:2788027812166856803, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.681+05:30 [Info] 		Index total memory:267781689109 (249.391G), mem:237221304305 (220.93G), overhead:30560384804 (28.4616G), min:25213825434 (23.4822G), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.681+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.681+05:30 [Info] 
2023-09-01T13:38:27.681+05:30 [Info] Indexer serverGroup:sg0, nodeId:127.0.0.1:9003, nodeUUID:934d23a204d25a4970e0262fa63920ba, useLiveData:true
2023-09-01T13:38:27.681+05:30 [Info] Indexer total memory:339383238621 (316.075G), mem:285379619397 (265.78G), overhead:54003619224 (50.2948G), min:50807128563 (47.3178G), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:2
2023-09-01T13:38:27.681+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0.2138043366147638 allowDDLDuringScaleup:true
2023-09-01T13:38:27.681+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.681+05:30 [Info] 		Index name:replicaIdx2 0, bucket:default, scope:, collection:, defnId:4242, instId:2, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.681+05:30 [Info] 		Index total memory:71601549512 (66.6841G), mem:48158315092 (44.8509G), overhead:23443234420 (21.8332G), min:25593303129 (23.8356G), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.681+05:30 [Info] 		Index resident:0% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.681+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.681+05:30 [Info] 		Index name:replicaIdx1 0 (replica 1), bucket:default, scope:, collection:, defnId:1337, instId:15426024519747553870, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.681+05:30 [Info] 		Index total memory:267781689109 (249.391G), mem:237221304305 (220.93G), overhead:30560384804 (28.4616G), min:25213825434 (23.4822G), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.681+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023/09/01 13:38:27 -------------------------------------------
2023/09/01 13:38:27 Increment Replica count by 1 - 3 non-empty heterogenous nodes with 2 Scaled up
2023-09-01T13:38:27.682+05:30 [Info] Rebuilding lost replica for (default,,,replicaIdx2,2,0) with initial placement on node: 127.0.0.1:9001
2023-09-01T13:38:27.682+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=1.  Elapsed Time=8us
2023-09-01T13:38:27.682+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=2.  Elapsed Time=9us
2023-09-01T13:38:27.682+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=3.  Elapsed Time=11us
2023-09-01T13:38:27.682+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=4.  Elapsed Time=9us
2023-09-01T13:38:27.682+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.6987684835628951
2023-09-01T13:38:27.682+05:30 [Info] Planner::initial temperature: initial resource variation 0.6987684835628951 temp 0.06987684835628952
2023-09-01T13:38:27.682+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (0, 0) exceeded.
2023-09-01T13:38:27.682+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:27.682+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=5.  Elapsed Time=176us
2023-09-01T13:38:27.683+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.6987684835628951
2023-09-01T13:38:27.683+05:30 [Info] Planner::initial temperature: initial resource variation 0.6987684835628951 temp 0.06987684835628952
2023-09-01T13:38:27.683+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (0, 0) exceeded.
2023-09-01T13:38:27.683+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:27.683+05:30 [Info] Score: 0.17469212089072378
2023-09-01T13:38:27.683+05:30 [Info] variation: 0.6987684835628951
2023-09-01T13:38:27.683+05:30 [Info] ElapsedTime: 104us
2023-09-01T13:38:27.683+05:30 [Info] ConvergenceTime: 3us
2023-09-01T13:38:27.683+05:30 [Info] Iteration: 0
2023-09-01T13:38:27.683+05:30 [Info] Move: 0
2023-09-01T13:38:27.683+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.683+05:30 [Info] Using SAPlanner
2023-09-01T13:38:27.683+05:30 [Info] Indexer Memory Mean 231925826675 (215.998G)
2023-09-01T13:38:27.683+05:30 [Info] Indexer Memory Deviation 162062458205 (150.932G) (69.88%)
2023-09-01T13:38:27.683+05:30 [Info] Indexer Memory Utilization 4.2047
2023-09-01T13:38:27.683+05:30 [Info] Indexer CPU Mean 0.0000
2023-09-01T13:38:27.683+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2023-09-01T13:38:27.683+05:30 [Info] Indexer CPU Utilization 0.0000
2023-09-01T13:38:27.683+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:27.683+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:27.683+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:27.683+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:27.683+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:27.683+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:27.683+05:30 [Info] Indexer Data Size Mean 0 (0)
2023-09-01T13:38:27.683+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2023-09-01T13:38:27.683+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:27.683+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:27.683+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:27.683+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:27.683+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.683+05:30 [Info] Number of indexes: 4
2023-09-01T13:38:27.683+05:30 [Info] Number of indexers: 3
2023-09-01T13:38:27.683+05:30 [Info] Avg Index Size: 173944370006 (161.998G)
2023-09-01T13:38:27.683+05:30 [Info] Max Index Size: 237221304305 (220.93G)
2023-09-01T13:38:27.683+05:30 [Info] Max Indexer Overhead: 49341212983 (45.9526G)
2023-09-01T13:38:27.683+05:30 [Info] Avg Index Cpu: 0.0000
2023-09-01T13:38:27.683+05:30 [Info] Max Index Cpu: 0.0000
2023-09-01T13:38:27.683+05:30 [Info] Num Estimation Run: 0
2023-09-01T13:38:27.683+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.683+05:30 [Info] Memory Quota 55158315092 (51.3702G)
2023-09-01T13:38:27.683+05:30 [Info] CPU Quota 27
2023-09-01T13:38:27.683+05:30 [Info] Max Cpu Utilization -1
2023-09-01T13:38:27.683+05:30 [Info] Max Memory Utilization -1
2023-09-01T13:38:27.683+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.683+05:30 [Info] Total iteration: 0
2023-09-01T13:38:27.683+05:30 [Info] RandomSwap time: 0ns
2023-09-01T13:38:27.683+05:30 [Info] RandomSwap call: 0
2023-09-01T13:38:27.683+05:30 [Info] RandomSwap iteration: 0
2023-09-01T13:38:27.683+05:30 [Info] RandomMove time: 0ns
2023-09-01T13:38:27.683+05:30 [Info] RandomMove call: 0 (empty index 0)
2023-09-01T13:38:27.683+05:30 [Info] ExhaustMove time: 0ns
2023-09-01T13:38:27.683+05:30 [Info] ExhaustMove call: 0
2023-09-01T13:38:27.683+05:30 [Info] ExhaustSwap time: 0ns
2023-09-01T13:38:27.683+05:30 [Info] ExhaustSwap call: 0
2023-09-01T13:38:27.683+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.683+05:30 [Info] 
2023-09-01T13:38:27.683+05:30 [Info] Indexer serverGroup:sg0, nodeId:127.0.0.1:9001, nodeUUID:af33bb1cf058349f6fafffa6a843a979, useLiveData:true
2023-09-01T13:38:27.683+05:30 [Info] Indexer total memory:491761447603 (457.989G), mem:442420234620 (412.036G), overhead:49341212983 (45.9526G), min:52248944865 (48.6606G), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:2
2023-09-01T13:38:27.683+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.38151873032984485 allowDDLDuringScaleup:true
2023-09-01T13:38:27.684+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.684+05:30 [Info] 		Index name:replicaIdx1 0, bucket:default, scope:, collection:, defnId:1337, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.684+05:30 [Info] 		Index total memory:267781689109 (249.391G), mem:237221304305 (220.93G), overhead:30560384804 (28.4616G), min:28015361594 (26.0913G), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.684+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.684+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.684+05:30 [Info] 		Index name:replicaIdx2 0 (replica 2), bucket:default, scope:, collection:, defnId:4242, instId:17425567927221601972, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.684+05:30 [Info] 		Index total memory:223979758494 (208.597G), mem:205198930315 (191.106G), overhead:18780828179 (17.491G), min:24233583271 (22.5693G), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.684+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.684+05:30 [Info] 
2023-09-01T13:38:27.684+05:30 [Info] Indexer serverGroup:sg0, nodeId:127.0.0.1:9002, nodeUUID:929bb73f8aaeb55422bdc071cf2dfbfc, useLiveData:true
2023-09-01T13:38:27.684+05:30 [Info] Indexer total memory:223979758494 (208.597G), mem:205198930315 (191.106G), overhead:18780828179 (17.491G), min:24233583271 (22.5693G), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2023-09-01T13:38:27.684+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.1769522034317959 allowDDLDuringScaleup:true
2023-09-01T13:38:27.684+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.684+05:30 [Info] 		Index name:replicaIdx2 0 (replica 1), bucket:default, scope:, collection:, defnId:4242, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.684+05:30 [Info] 		Index total memory:223979758494 (208.597G), mem:205198930315 (191.106G), overhead:18780828179 (17.491G), min:24233583271 (22.5693G), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.684+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.684+05:30 [Info] 
2023-09-01T13:38:27.684+05:30 [Info] Indexer serverGroup:sg0, nodeId:127.0.0.1:9003, nodeUUID:934d23a204d25a4970e0262fa63920ba, useLiveData:true
2023-09-01T13:38:27.684+05:30 [Info] Indexer total memory:71601549512 (66.6841G), mem:48158315092 (44.8509G), overhead:23443234420 (21.8332G), min:28437003477 (26.484G), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2023-09-01T13:38:27.684+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0.0415290662383593 allowDDLDuringScaleup:true
2023-09-01T13:38:27.684+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.684+05:30 [Info] 		Index name:replicaIdx2 0, bucket:default, scope:, collection:, defnId:4242, instId:2, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.684+05:30 [Info] 		Index total memory:71601549512 (66.6841G), mem:48158315092 (44.8509G), overhead:23443234420 (21.8332G), min:28437003477 (26.484G), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.684+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023/09/01 13:38:27 -------------------------------------------
2023/09/01 13:38:27 Increment Replica count by 2 - 3 non-empty heterogenous nodes with 2 Scaled up
2023-09-01T13:38:27.684+05:30 [Info] Rebuilding lost replica for (default,,,replicaIdx1,1,0) with initial placement on node: 127.0.0.1:9003
2023-09-01T13:38:27.684+05:30 [Info] Rebuilding lost replica for (default,,,replicaIdx1,2,0) with initial placement on node: 127.0.0.1:9002
2023-09-01T13:38:27.684+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=1.  Elapsed Time=7us
2023-09-01T13:38:27.684+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=2.  Elapsed Time=8us
2023-09-01T13:38:27.684+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=3.  Elapsed Time=6us
2023-09-01T13:38:27.685+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=4.  Elapsed Time=6us
2023-09-01T13:38:27.685+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=5.  Elapsed Time=9us
2023-09-01T13:38:27.685+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.27237243417194856
2023-09-01T13:38:27.685+05:30 [Info] Planner::initial temperature: initial resource variation 0.27237243417194856 temp 0.027237243417194856
2023-09-01T13:38:27.685+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (0, 0) exceeded.
2023-09-01T13:38:27.685+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:27.685+05:30 [Info] Planner::Fail to create plan satisfying constraint. Re-planning. Num of Try=6.  Elapsed Time=174us
2023-09-01T13:38:27.685+05:30 [Warn] Unable to find a solution with resource constraint.  Relax resource constraint check.
2023-09-01T13:38:27.685+05:30 [Info] Planner::planSingleRun Initial variance of the solution: 0.27237243417194856
2023-09-01T13:38:27.685+05:30 [Info] Planner::initial temperature: initial resource variation 0.27237243417194856 temp 0.027237243417194856
2023-09-01T13:38:27.685+05:30 [Info] Planner::stop planner iter per temp as maxIterPerTemp limit (0, 0) exceeded.
2023-09-01T13:38:27.685+05:30 [Info] Planner::finalizing the solution as there are no more valid index movements.
2023-09-01T13:38:27.685+05:30 [Info] Score: 0.06809310854298714
2023-09-01T13:38:27.685+05:30 [Info] variation: 0.27237243417194856
2023-09-01T13:38:27.685+05:30 [Info] ElapsedTime: 97us
2023-09-01T13:38:27.685+05:30 [Info] ConvergenceTime: 2us
2023-09-01T13:38:27.685+05:30 [Info] Iteration: 0
2023-09-01T13:38:27.685+05:30 [Info] Move: 0
2023-09-01T13:38:27.685+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.685+05:30 [Info] Using SAPlanner
2023-09-01T13:38:27.685+05:30 [Info] Indexer Memory Mean 321673719440 (299.582G)
2023-09-01T13:38:27.685+05:30 [Info] Indexer Memory Deviation 87615053973 (81.5979G) (27.24%)
2023-09-01T13:38:27.685+05:30 [Info] Indexer Memory Utilization 5.8318
2023-09-01T13:38:27.685+05:30 [Info] Indexer CPU Mean 0.0000
2023-09-01T13:38:27.685+05:30 [Info] Indexer CPU Deviation 0.00 (0.00%)
2023-09-01T13:38:27.685+05:30 [Info] Indexer CPU Utilization 0.0000
2023-09-01T13:38:27.685+05:30 [Info] Indexer IO Mean 0.0000
2023-09-01T13:38:27.685+05:30 [Info] Indexer IO Deviation 0.00 (0.00%)
2023-09-01T13:38:27.685+05:30 [Info] Indexer Drain Rate Mean 0.0000
2023-09-01T13:38:27.685+05:30 [Info] Indexer Drain Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:27.685+05:30 [Info] Indexer Scan Rate Mean 0.0000
2023-09-01T13:38:27.685+05:30 [Info] Indexer Scan Rate Deviation 0.00 (0.00%)
2023-09-01T13:38:27.685+05:30 [Info] Indexer Data Size Mean 0 (0)
2023-09-01T13:38:27.685+05:30 [Info] Indexer Data Size Deviation 0 (0) (0.00%)
2023-09-01T13:38:27.685+05:30 [Info] Total Index Data (from non-deleted node) 0
2023-09-01T13:38:27.685+05:30 [Info] Index Data Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:27.685+05:30 [Info] No. Index (from non-deleted node) 0
2023-09-01T13:38:27.685+05:30 [Info] No. Index Moved (exclude new node) 0 (0.00%)
2023-09-01T13:38:27.685+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.685+05:30 [Info] Number of indexes: 5
2023-09-01T13:38:27.685+05:30 [Info] Number of indexers: 3
2023-09-01T13:38:27.685+05:30 [Info] Avg Index Size: 193004231664 (179.749G)
2023-09-01T13:38:27.685+05:30 [Info] Max Index Size: 237221304305 (220.93G)
2023-09-01T13:38:27.685+05:30 [Info] Max Indexer Overhead: 54003619224 (50.2948G)
2023-09-01T13:38:27.685+05:30 [Info] Avg Index Cpu: 0.0000
2023-09-01T13:38:27.685+05:30 [Info] Max Index Cpu: 0.0000
2023-09-01T13:38:27.685+05:30 [Info] Num Estimation Run: 0
2023-09-01T13:38:27.685+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.685+05:30 [Info] Memory Quota 55158315092 (51.3702G)
2023-09-01T13:38:27.685+05:30 [Info] CPU Quota 27
2023-09-01T13:38:27.686+05:30 [Info] Max Cpu Utilization -1
2023-09-01T13:38:27.686+05:30 [Info] Max Memory Utilization -1
2023-09-01T13:38:27.686+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.686+05:30 [Info] Total iteration: 0
2023-09-01T13:38:27.686+05:30 [Info] RandomSwap time: 0ns
2023-09-01T13:38:27.686+05:30 [Info] RandomSwap call: 0
2023-09-01T13:38:27.686+05:30 [Info] RandomSwap iteration: 0
2023-09-01T13:38:27.686+05:30 [Info] RandomMove time: 0ns
2023-09-01T13:38:27.686+05:30 [Info] RandomMove call: 0 (empty index 0)
2023-09-01T13:38:27.686+05:30 [Info] ExhaustMove time: 0ns
2023-09-01T13:38:27.686+05:30 [Info] ExhaustMove call: 0
2023-09-01T13:38:27.686+05:30 [Info] ExhaustSwap time: 0ns
2023-09-01T13:38:27.686+05:30 [Info] ExhaustSwap call: 0
2023-09-01T13:38:27.686+05:30 [Info] ----------------------------------------
2023-09-01T13:38:27.686+05:30 [Info] 
2023-09-01T13:38:27.686+05:30 [Info] Indexer serverGroup:sg0, nodeId:127.0.0.1:9001, nodeUUID:af33bb1cf058349f6fafffa6a843a979, useLiveData:true
2023-09-01T13:38:27.686+05:30 [Info] Indexer total memory:267781689109 (249.391G), mem:237221304305 (220.93G), overhead:30560384804 (28.4616G), min:25213825434 (23.4822G), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:1
2023-09-01T13:38:27.686+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.14749187761902688 allowDDLDuringScaleup:true
2023-09-01T13:38:27.686+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.686+05:30 [Info] 		Index name:replicaIdx1 0, bucket:default, scope:, collection:, defnId:1337, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.686+05:30 [Info] 		Index total memory:267781689109 (249.391G), mem:237221304305 (220.93G), overhead:30560384804 (28.4616G), min:25213825434 (23.4822G), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.686+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.686+05:30 [Info] 
2023-09-01T13:38:27.686+05:30 [Info] Indexer serverGroup:sg0, nodeId:127.0.0.1:9002, nodeUUID:929bb73f8aaeb55422bdc071cf2dfbfc, useLiveData:true
2023-09-01T13:38:27.686+05:30 [Info] Indexer total memory:491761447603 (457.989G), mem:442420234620 (412.036G), overhead:49341212983 (45.9526G), min:47024050377 (43.7946G), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:2
2023-09-01T13:38:27.686+05:30 [Info] Indexer isDeleted:false isNew:false exclude:inout meetConstraint:true usageRatio:0.2750739074297335 allowDDLDuringScaleup:true
2023-09-01T13:38:27.686+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.686+05:30 [Info] 		Index name:replicaIdx2 0 (replica 1), bucket:default, scope:, collection:, defnId:4242, instId:1, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.686+05:30 [Info] 		Index total memory:223979758494 (208.597G), mem:205198930315 (191.106G), overhead:18780828179 (17.491G), min:21810224943 (20.3124G), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.686+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.686+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.686+05:30 [Info] 		Index name:replicaIdx1 0 (replica 2), bucket:default, scope:, collection:, defnId:1337, instId:8609891115038844288, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.686+05:30 [Info] 		Index total memory:267781689109 (249.391G), mem:237221304305 (220.93G), overhead:30560384804 (28.4616G), min:25213825434 (23.4822G), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.686+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.686+05:30 [Info] 
2023-09-01T13:38:27.686+05:30 [Info] Indexer serverGroup:sg0, nodeId:127.0.0.1:9003, nodeUUID:934d23a204d25a4970e0262fa63920ba, useLiveData:true
2023-09-01T13:38:27.686+05:30 [Info] Indexer total memory:339383238621 (316.075G), mem:285379619397 (265.78G), overhead:54003619224 (50.2948G), min:50807128563 (47.3178G), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:2
2023-09-01T13:38:27.686+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0.17743421495123962 allowDDLDuringScaleup:true
2023-09-01T13:38:27.686+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.686+05:30 [Info] 		Index name:replicaIdx2 0, bucket:default, scope:, collection:, defnId:4242, instId:2, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.686+05:30 [Info] 		Index total memory:71601549512 (66.6841G), mem:48158315092 (44.8509G), overhead:23443234420 (21.8332G), min:25593303129 (23.8356G), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.686+05:30 [Info] 		Index resident:10% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
2023-09-01T13:38:27.686+05:30 [Info] 		------------------------------------------------------------------------------------------------------------------
2023-09-01T13:38:27.686+05:30 [Info] 		Index name:replicaIdx1 0 (replica 1), bucket:default, scope:, collection:, defnId:1337, instId:975293282400869354, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: []
2023-09-01T13:38:27.686+05:30 [Info] 		Index total memory:267781689109 (249.391G), mem:237221304305 (220.93G), overhead:30560384804 (28.4616G), min:25213825434 (23.4822G), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0
2023-09-01T13:38:27.686+05:30 [Info] 		Index resident:50% build:0% estimated:false equivCheck:true pendingCreate:false pendingDelete:false
--- PASS: TestPlanDuringHeterogenousScaleup (1.16s)
    --- PASS: TestPlanDuringHeterogenousScaleup/GreedyPlanner (0.02s)
    --- PASS: TestPlanDuringHeterogenousScaleup/SAPlanner (1.13s)
        --- PASS: TestPlanDuringHeterogenousScaleup/SAPlanner/FunctionalDDLTests (0.40s)
        --- PASS: TestPlanDuringHeterogenousScaleup/SAPlanner/CreatePartitionedIndex (0.73s)
    --- PASS: TestPlanDuringHeterogenousScaleup/CommandRepair (0.01s)
=== RUN   TestRestfulAPI
2023/09/01 13:38:27 In TestRestfulAPI()
2023/09/01 13:38:27 In DropAllSecondaryIndexes()
2023/09/01 13:38:27 Index found:  indexmut_1
2023/09/01 13:38:27 Dropped index indexmut_1
2023/09/01 13:38:28 Setting JSON docs in KV
2023/09/01 13:38:28 GET all indexes
2023/09/01 13:38:28 200 OK
2023/09/01 13:38:28 FOUND indexes: []
2023/09/01 13:38:28 DROP index: badindexid
2023/09/01 13:38:28 status: 400 Bad Request
2023/09/01 13:38:28 DROP index: 23544142
2023/09/01 13:38:28 status: 500 Internal Server Error
2023/09/01 13:38:28 TEST: malformed body
2023/09/01 13:38:28 400 Bad Request "invalid request body ({name:), unmarshal failed invalid character 'n' looking for beginning of object key string"

2023/09/01 13:38:28 TEST: missing field ``name``
2023/09/01 13:38:28 400 Bad Request "missing field name"
2023/09/01 13:38:28 TEST: empty field ``name``
2023/09/01 13:38:28 400 Bad Request "empty field name"
2023/09/01 13:38:28 TEST: missing field ``bucket``
2023/09/01 13:38:28 400 Bad Request "missing field bucket"
2023/09/01 13:38:28 TEST: empty field ``bucket``
2023/09/01 13:38:28 400 Bad Request "empty field bucket"
2023/09/01 13:38:28 TEST: missing field ``secExprs``
2023/09/01 13:38:28 400 Bad Request "missing field secExprs"
2023/09/01 13:38:28 TEST: empty field ``secExprs``
2023/09/01 13:38:28 400 Bad Request "empty field secExprs"
2023/09/01 13:38:28 TEST: incomplete field ``desc``
2023/09/01 13:38:28 400 Bad Request "incomplete desc information [true]"
2023/09/01 13:38:28 TEST: invalid field ``desc``
2023/09/01 13:38:28 400 Bad Request "incomplete desc information [1]"
2023/09/01 13:38:28 
2023/09/01 13:38:28 CREATE INDEX: idx1
2023/09/01 13:38:41 status : 201 Created
2023/09/01 13:38:41 {"id": "14187258152876684404"} 
2023/09/01 13:38:41 CREATE INDEX: idx2 (defer)
2023/09/01 13:38:41 status : 201 Created
2023/09/01 13:38:41 {"id": "5302126075343952547"} 
2023/09/01 13:38:41 CREATE INDEX: idx3 (defer)
2023/09/01 13:38:41 status : 201 Created
2023/09/01 13:38:41 {"id": "17268260782802194534"} 
2023/09/01 13:38:41 CREATE INDEX: idx4 (defer)
2023/09/01 13:38:41 status : 201 Created
2023/09/01 13:38:41 {"id": "15870127287913189436"} 
2023/09/01 13:38:41 CREATE INDEX: idx5
2023/09/01 13:38:54 status : 201 Created
2023/09/01 13:38:54 {"id": "313943933983161517"} 
2023/09/01 13:38:54 BUILD single deferred index
2023/09/01 13:38:54 202 Accepted
2023/09/01 13:38:54 GET all indexes
2023/09/01 13:38:54 200 OK
2023/09/01 13:38:54 index idx1 in INDEX_STATE_ACTIVE
2023/09/01 13:38:54 GET all indexes
2023/09/01 13:38:54 200 OK
2023/09/01 13:38:54 index idx2 in INDEX_STATE_INITIAL
2023/09/01 13:38:55 GET all indexes
2023/09/01 13:38:55 200 OK
2023/09/01 13:38:55 index idx2 in INDEX_STATE_INITIAL
2023/09/01 13:38:56 GET all indexes
2023/09/01 13:38:56 200 OK
2023/09/01 13:38:56 index idx2 in INDEX_STATE_INITIAL
2023/09/01 13:38:57 GET all indexes
2023/09/01 13:38:57 200 OK
2023/09/01 13:38:57 index idx2 in INDEX_STATE_INITIAL
2023/09/01 13:38:59 GET all indexes
2023/09/01 13:38:59 200 OK
2023/09/01 13:38:59 index idx2 in INDEX_STATE_INITIAL
2023/09/01 13:39:00 GET all indexes
2023/09/01 13:39:00 200 OK
2023/09/01 13:39:00 index idx2 in INDEX_STATE_INITIAL
2023/09/01 13:39:01 GET all indexes
2023/09/01 13:39:01 200 OK
2023/09/01 13:39:01 index idx2 in INDEX_STATE_INITIAL
2023/09/01 13:39:02 GET all indexes
2023/09/01 13:39:02 200 OK
2023/09/01 13:39:02 index idx2 in INDEX_STATE_INITIAL
2023/09/01 13:39:03 GET all indexes
2023/09/01 13:39:03 200 OK
2023/09/01 13:39:03 index idx2 in INDEX_STATE_INITIAL
2023/09/01 13:39:04 GET all indexes
2023/09/01 13:39:04 200 OK
2023/09/01 13:39:04 index idx2 in INDEX_STATE_INITIAL
2023/09/01 13:39:05 GET all indexes
2023/09/01 13:39:05 200 OK
2023/09/01 13:39:05 index idx2 in INDEX_STATE_INITIAL
2023/09/01 13:39:06 GET all indexes
2023/09/01 13:39:06 200 OK
2023/09/01 13:39:06 index idx2 in INDEX_STATE_INITIAL
2023/09/01 13:39:07 GET all indexes
2023/09/01 13:39:07 200 OK
2023/09/01 13:39:07 index idx2 in INDEX_STATE_INITIAL
2023/09/01 13:39:08 GET all indexes
2023/09/01 13:39:08 200 OK
2023/09/01 13:39:08 index idx2 in INDEX_STATE_INITIAL
2023/09/01 13:39:09 GET all indexes
2023/09/01 13:39:09 200 OK
2023/09/01 13:39:09 index idx2 in INDEX_STATE_ACTIVE
2023/09/01 13:39:09 BUILD many deferred index
2023/09/01 13:39:09 202 Accepted 
2023/09/01 13:39:09 GET all indexes
2023/09/01 13:39:09 200 OK
2023/09/01 13:39:09 index idx1 in INDEX_STATE_ACTIVE
2023/09/01 13:39:09 GET all indexes
2023/09/01 13:39:09 200 OK
2023/09/01 13:39:09 index idx2 in INDEX_STATE_ACTIVE
2023/09/01 13:39:09 GET all indexes
2023/09/01 13:39:09 200 OK
2023/09/01 13:39:09 index idx3 in INDEX_STATE_INITIAL
2023/09/01 13:39:11 GET all indexes
2023/09/01 13:39:11 200 OK
2023/09/01 13:39:11 index idx3 in INDEX_STATE_INITIAL
2023/09/01 13:39:12 GET all indexes
2023/09/01 13:39:12 200 OK
2023/09/01 13:39:12 index idx3 in INDEX_STATE_INITIAL
2023/09/01 13:39:13 GET all indexes
2023/09/01 13:39:13 200 OK
2023/09/01 13:39:13 index idx3 in INDEX_STATE_INITIAL
2023/09/01 13:39:14 GET all indexes
2023/09/01 13:39:14 200 OK
2023/09/01 13:39:14 index idx3 in INDEX_STATE_INITIAL
2023/09/01 13:39:15 GET all indexes
2023/09/01 13:39:15 200 OK
2023/09/01 13:39:15 index idx3 in INDEX_STATE_INITIAL
2023/09/01 13:39:16 GET all indexes
2023/09/01 13:39:16 200 OK
2023/09/01 13:39:16 index idx3 in INDEX_STATE_INITIAL
2023/09/01 13:39:17 GET all indexes
2023/09/01 13:39:17 200 OK
2023/09/01 13:39:17 index idx3 in INDEX_STATE_INITIAL
2023/09/01 13:39:18 GET all indexes
2023/09/01 13:39:18 200 OK
2023/09/01 13:39:18 index idx3 in INDEX_STATE_INITIAL
2023/09/01 13:39:19 GET all indexes
2023/09/01 13:39:19 200 OK
2023/09/01 13:39:19 index idx3 in INDEX_STATE_INITIAL
2023/09/01 13:39:20 GET all indexes
2023/09/01 13:39:20 200 OK
2023/09/01 13:39:20 index idx3 in INDEX_STATE_INITIAL
2023/09/01 13:39:21 GET all indexes
2023/09/01 13:39:21 200 OK
2023/09/01 13:39:21 index idx3 in INDEX_STATE_INITIAL
2023/09/01 13:39:22 GET all indexes
2023/09/01 13:39:22 200 OK
2023/09/01 13:39:22 index idx3 in INDEX_STATE_INITIAL
2023/09/01 13:39:23 GET all indexes
2023/09/01 13:39:24 200 OK
2023/09/01 13:39:24 index idx3 in INDEX_STATE_INITIAL
2023/09/01 13:39:25 GET all indexes
2023/09/01 13:39:25 200 OK
2023/09/01 13:39:25 index idx3 in INDEX_STATE_INITIAL
2023/09/01 13:39:26 GET all indexes
2023/09/01 13:39:26 200 OK
2023/09/01 13:39:26 index idx3 in INDEX_STATE_INITIAL
2023/09/01 13:39:27 GET all indexes
2023/09/01 13:39:27 200 OK
2023/09/01 13:39:27 index idx3 in INDEX_STATE_INITIAL
2023/09/01 13:39:28 GET all indexes
2023/09/01 13:39:28 200 OK
2023/09/01 13:39:28 index idx3 in INDEX_STATE_INITIAL
2023/09/01 13:39:29 GET all indexes
2023/09/01 13:39:29 200 OK
2023/09/01 13:39:29 index idx3 in INDEX_STATE_ACTIVE
2023/09/01 13:39:29 GET all indexes
2023/09/01 13:39:29 200 OK
2023/09/01 13:39:29 index idx4 in INDEX_STATE_ACTIVE
2023/09/01 13:39:29 GET all indexes
2023/09/01 13:39:29 200 OK
2023/09/01 13:39:29 index idx5 in INDEX_STATE_ACTIVE
2023/09/01 13:39:29 GET all indexes
2023/09/01 13:39:29 200 OK
2023/09/01 13:39:29 CREATED indexes: [14187258152876684404 5302126075343952547 17268260782802194534 15870127287913189436 313943933983161517]
2023/09/01 13:39:29 
2023/09/01 13:39:29 LOOKUP missing index
2023/09/01 13:39:29 status : 404 Not Found
2023/09/01 13:39:29 LOOKUP Pyongyang
2023/09/01 13:39:29 status : 200 OK
2023/09/01 13:39:29 number of entries 554
2023/09/01 13:39:29 Expected and Actual scan responses are the same
2023/09/01 13:39:29 LOOKUP with stale as false
2023/09/01 13:39:29 status : 200 OK
2023/09/01 13:39:29 number of entries 554
2023/09/01 13:39:29 Expected and Actual scan responses are the same
2023/09/01 13:39:29 LOOKUP with Rome
2023/09/01 13:39:29 status : 200 OK
2023/09/01 13:39:29 number of entries 540
2023/09/01 13:39:29 Expected and Actual scan responses are the same
2023/09/01 13:39:29 RANGE missing index
2023/09/01 13:39:29 Status : 404 Not Found
2023/09/01 13:39:29 RANGE cities - none
2023/09/01 13:39:29 Status : 200 OK
2023/09/01 13:39:35 number of entries 140902
2023/09/01 13:39:35 Expected and Actual scan responses are the same
2023/09/01 13:39:35 RANGE cities -low
2023/09/01 13:39:35 Status : 200 OK
2023/09/01 13:39:40 number of entries 140902
2023/09/01 13:39:41 Expected and Actual scan responses are the same
2023/09/01 13:39:41 RANGE cities -high
2023/09/01 13:39:41 Status : 200 OK
2023/09/01 13:39:45 number of entries 140902
2023/09/01 13:39:47 Expected and Actual scan responses are the same
2023/09/01 13:39:47 RANGE cities - both
2023/09/01 13:39:47 Status : 200 OK
2023/09/01 13:39:51 number of entries 140902
2023/09/01 13:39:52 Expected and Actual scan responses are the same
2023/09/01 13:39:52 RANGE missing cities
2023/09/01 13:39:52 Status : 200 OK
2023/09/01 13:39:52 number of entries 0
2023/09/01 13:39:52 Expected and Actual scan responses are the same
2023/09/01 13:39:52 
2023/09/01 13:39:52 SCANALL missing index
2023/09/01 13:39:52 {"limit":1000000,"stale":"ok"}
2023/09/01 13:39:52 Status : 404 Not Found
2023/09/01 13:39:52 SCANALL stale ok
2023/09/01 13:39:52 {"limit":1000000,"stale":"ok"}
2023/09/01 13:39:52 Status : 200 OK
2023/09/01 13:39:57 number of entries 140902
2023/09/01 13:39:58 Expected and Actual scan responses are the same
2023/09/01 13:39:58 SCANALL stale false
2023/09/01 13:39:58 {"limit":1000000,"stale":"false"}
2023/09/01 13:39:58 Status : 200 OK
2023/09/01 13:40:03 number of entries 140902
2023/09/01 13:40:04 Expected and Actual scan responses are the same
2023/09/01 13:40:04 
2023/09/01 13:40:04 COUNT missing index
2023/09/01 13:40:04 Status : 404 Not Found
2023/09/01 13:40:04 COUNT cities - none
2023/09/01 13:40:04 Status : 200 OK
2023/09/01 13:40:04 number of entries 140902
2023/09/01 13:40:04 COUNT cities -low
2023/09/01 13:40:04 Status : 200 OK
2023/09/01 13:40:04 number of entries 140902
2023/09/01 13:40:05 COUNT cities -high
2023/09/01 13:40:05 Status : 200 OK
2023/09/01 13:40:05 number of entries 140902
2023/09/01 13:40:05 COUNT cities - both
2023/09/01 13:40:05 Status : 200 OK
2023/09/01 13:40:05 number of entries 140902
2023/09/01 13:40:05 COUNT missing cities
2023/09/01 13:40:05 Status : 200 OK
2023/09/01 13:40:05 number of entries 0
2023/09/01 13:40:06 
2023/09/01 13:40:07 STATS: Testing URLs with valid authentication
2023/09/01 13:40:07 STATS: Testing URLs with invalid authentication
2023/09/01 13:40:07 STATS: Testing invalid URLs
2023/09/01 13:40:08 STATS: Testing unsupported methods
2023/09/01 13:40:08 
--- PASS: TestRestfulAPI (100.34s)
=== RUN   TestStatIndexInstFilter
2023/09/01 13:40:08 CREATE INDEX: statIdx1
2023/09/01 13:40:20 status : 201 Created
2023/09/01 13:40:20 {"id": "363069625326855071"} 
2023/09/01 13:40:20 CREATE INDEX: statIdx2
2023/09/01 13:40:34 status : 201 Created
2023/09/01 13:40:34 {"id": "11242552597240971935"} 
2023/09/01 13:40:34 Instance Id for statIdx2 is 7665738774927741091, common.IndexInstId
--- PASS: TestStatIndexInstFilter (26.61s)
=== RUN   TestBucketDefaultDelete
2023-09-01T13:40:34.763+05:30 [Warn] Client:runObserveStreamingEndpoint streaming endpoint for /pools/default/bs/default returned err EOF
2023-09-01T13:40:34.763+05:30 [Warn] serviceChangeNotifier: Connection terminated for collection manifest notifier instance of http://%40query@127.0.0.1:9000-default, bucket: default, (EOF)
2023/09/01 13:40:37 Deleted bucket default, responseBody: 
2023/09/01 13:40:52 Created bucket default, responseBody: 
2023/09/01 13:41:08 Populating the default bucket
2023/09/01 13:41:15 Using n1ql client
2023-09-01T13:41:15.974+05:30 [Info] GSIC[default/default-_default-_default-1693555875972475449] started ...
2023/09/01 13:41:15 Scan failed as expected with error: Index Not Found - cause: GSI index index_isActive not found.
2023/09/01 13:41:15 Populating the default bucket after it was deleted
2023/09/01 13:41:25 Created the secondary index index_isActive. Waiting for it become active
2023/09/01 13:41:25 Index is 15978923376999005623 now active
2023/09/01 13:41:25 Using n1ql client
2023/09/01 13:41:25 Expected and Actual scan responses are the same
--- PASS: TestBucketDefaultDelete (50.96s)
=== RUN   TestMixedDatatypesScanAll
2023/09/01 13:41:25 In TestMixedDatatypesScanAll()
2023/09/01 13:41:25 Before test begin: Length of kv docs is 10002
2023/09/01 13:41:25 In DropAllSecondaryIndexes()
2023/09/01 13:41:25 Index found:  index_isActive
2023/09/01 13:41:25 Dropped index index_isActive
2023/09/01 13:41:25 Number of number fields is: 247
2023/09/01 13:41:25 Number of string fields is: 260
2023/09/01 13:41:25 Number of json fields is: 239
2023/09/01 13:41:25 Number of true bool fields is: 129
2023/09/01 13:41:25 Number of false bool fields is: 125
2023/09/01 13:41:25 After generate docs: Length of kv docs is 11002
2023/09/01 13:41:25 Setting mixed datatypes JSON docs in KV
2023/09/01 13:41:31 Created the secondary index index_mixeddt. Waiting for it become active
2023/09/01 13:41:31 Index is 3048001384186340268 now active
2023/09/01 13:41:31 Using n1ql client
2023/09/01 13:41:31 Expected and Actual scan responses are the same
2023/09/01 13:41:31 Lengths of expected and actual scan results are:  1000 and 1000
2023/09/01 13:41:31 End: Length of kv docs is 11002
--- PASS: TestMixedDatatypesScanAll (5.90s)
=== RUN   TestMixedDatatypesRange_Float
2023/09/01 13:41:31 In TestMixedDatatypesRange_Float()
2023/09/01 13:41:31 In DropAllSecondaryIndexes()
2023/09/01 13:41:31 Index found:  index_mixeddt
2023/09/01 13:41:31 Dropped index index_mixeddt
2023/09/01 13:41:31 Number of number fields is: 238
2023/09/01 13:41:31 Number of string fields is: 263
2023/09/01 13:41:31 Number of json fields is: 262
2023/09/01 13:41:31 Number of true bool fields is: 109
2023/09/01 13:41:31 Number of false bool fields is: 128
2023/09/01 13:41:31 Setting mixed datatypes JSON docs in KV
2023/09/01 13:41:37 Created the secondary index index_mixeddt. Waiting for it become active
2023/09/01 13:41:37 Index is 9712657337414230787 now active
2023/09/01 13:41:37 Using n1ql client
2023/09/01 13:41:37 Expected and Actual scan responses are the same
2023/09/01 13:41:37 Lengths of expected and actual scan results are:  15 and 15
2023/09/01 13:41:37 Using n1ql client
2023/09/01 13:41:37 Expected and Actual scan responses are the same
2023/09/01 13:41:37 Lengths of expected and actual scan results are:  3 and 3
2023/09/01 13:41:37 Length of kv docs is 12002
--- PASS: TestMixedDatatypesRange_Float (5.89s)
=== RUN   TestMixedDatatypesRange_String
2023/09/01 13:41:37 In TestMixedDatatypesRange_String()
2023/09/01 13:41:37 In DropAllSecondaryIndexes()
2023/09/01 13:41:37 Index found:  index_mixeddt
2023/09/01 13:41:37 Dropped index index_mixeddt
2023/09/01 13:41:37 Number of number fields is: 244
2023/09/01 13:41:37 Number of string fields is: 234
2023/09/01 13:41:37 Number of json fields is: 253
2023/09/01 13:41:37 Number of true bool fields is: 151
2023/09/01 13:41:37 Number of false bool fields is: 118
2023/09/01 13:41:37 Setting mixed datatypes JSON docs in KV
2023/09/01 13:41:43 Created the secondary index index_mixeddt. Waiting for it become active
2023/09/01 13:41:43 Index is 6523903957635483352 now active
2023/09/01 13:41:43 Using n1ql client
2023/09/01 13:41:43 Expected and Actual scan responses are the same
2023/09/01 13:41:43 Lengths of expected and actual scan results are:  186 and 186
2023/09/01 13:41:43 Length of kv docs is 13002
--- PASS: TestMixedDatatypesRange_String (5.97s)
=== RUN   TestMixedDatatypesRange_Json
2023/09/01 13:41:43 In TestMixedDatatypesRange_Json()
2023/09/01 13:41:43 In DropAllSecondaryIndexes()
2023/09/01 13:41:43 Index found:  index_mixeddt
2023/09/01 13:41:43 Dropped index index_mixeddt
2023/09/01 13:41:43 Number of number fields is: 248
2023/09/01 13:41:43 Number of string fields is: 250
2023/09/01 13:41:43 Number of json fields is: 251
2023/09/01 13:41:43 Number of true bool fields is: 136
2023/09/01 13:41:43 Number of false bool fields is: 115
2023/09/01 13:41:43 Setting mixed datatypes JSON docs in KV
2023/09/01 13:41:49 Created the secondary index index_mixeddt. Waiting for it become active
2023/09/01 13:41:49 Index is 827569476374620667 now active
2023/09/01 13:41:49 Using n1ql client
2023/09/01 13:41:49 Expected and Actual scan responses are the same
2023/09/01 13:41:49 Lengths of expected and actual scan results are:  766 and 766
2023/09/01 13:41:49 Length of kv docs is 14002
--- PASS: TestMixedDatatypesRange_Json (6.04s)
=== RUN   TestMixedDatatypesScan_Bool
2023/09/01 13:41:49 In TestMixedDatatypesScan_Bool()
2023/09/01 13:41:49 In DropAllSecondaryIndexes()
2023/09/01 13:41:49 Index found:  index_mixeddt
2023/09/01 13:41:49 Dropped index index_mixeddt
2023/09/01 13:41:49 Number of number fields is: 246
2023/09/01 13:41:49 Number of string fields is: 248
2023/09/01 13:41:49 Number of json fields is: 243
2023/09/01 13:41:49 Number of true bool fields is: 139
2023/09/01 13:41:49 Number of false bool fields is: 124
2023/09/01 13:41:49 Setting mixed datatypes JSON docs in KV
2023/09/01 13:41:55 Created the secondary index index_mixeddt. Waiting for it become active
2023/09/01 13:41:55 Index is 4791013999784639971 now active
2023/09/01 13:41:55 Using n1ql client
2023/09/01 13:41:55 Expected and Actual scan responses are the same
2023/09/01 13:41:55 Lengths of expected and actual scan results are:  535 and 535
2023/09/01 13:41:55 Using n1ql client
2023/09/01 13:41:55 Expected and Actual scan responses are the same
2023/09/01 13:41:55 Lengths of expected and actual scan results are:  485 and 485
2023/09/01 13:41:55 Length of kv docs is 15002
--- PASS: TestMixedDatatypesScan_Bool (6.07s)
=== RUN   TestLargeSecondaryKeyLength
2023/09/01 13:41:55 In TestLargeSecondaryKeyLength()
2023/09/01 13:41:55 In DropAllSecondaryIndexes()
2023/09/01 13:41:55 Index found:  index_mixeddt
2023/09/01 13:41:55 Dropped index index_mixeddt
2023/09/01 13:41:55 Setting JSON docs in KV
2023/09/01 13:42:01 Created the secondary index index_LongSecField. Waiting for it become active
2023/09/01 13:42:01 Index is 11571216158701766785 now active
2023/09/01 13:42:01 Using n1ql client
2023/09/01 13:42:01 ScanAll: Lengths of expected and actual scan results are:  1000 and 1000
2023/09/01 13:42:01 Expected and Actual scan responses are the same
2023/09/01 13:42:01 Using n1ql client
2023/09/01 13:42:01 Range: Lengths of expected and actual scan results are:  807 and 807
2023/09/01 13:42:01 Expected and Actual scan responses are the same
2023/09/01 13:42:01 End: Length of kv docs is 16002
--- PASS: TestLargeSecondaryKeyLength (6.45s)
=== RUN   TestLargePrimaryKeyLength
2023/09/01 13:42:01 In TestLargePrimaryKeyLength()
2023/09/01 13:42:01 In DropAllSecondaryIndexes()
2023/09/01 13:42:01 Index found:  index_LongSecField
2023/09/01 13:42:01 Dropped index index_LongSecField
2023/09/01 13:42:02 Setting JSON docs in KV
2023/09/01 13:42:08 Created the secondary index index_LongPrimaryField. Waiting for it become active
2023/09/01 13:42:08 Index is 12750094570884671025 now active
2023/09/01 13:42:08 Using n1ql client
2023/09/01 13:42:08 Lengths of num of docs and scanResults are:  17002 and 17002
2023/09/01 13:42:08 End: Length of kv docs is 17002
--- PASS: TestLargePrimaryKeyLength (6.83s)
=== RUN   TestUpdateMutations_DeleteField
2023/09/01 13:42:08 In TestUpdateMutations_DeleteField()
2023/09/01 13:42:09 Setting JSON docs in KV
2023/09/01 13:42:16 Created the secondary index index_bal. Waiting for it become active
2023/09/01 13:42:16 Index is 7409646789830203862 now active
2023/09/01 13:42:16 Using n1ql client
2023/09/01 13:42:16 Expected and Actual scan responses are the same
2023/09/01 13:42:16 Using n1ql client
2023/09/01 13:42:17 Expected and Actual scan responses are the same
--- PASS: TestUpdateMutations_DeleteField (8.30s)
=== RUN   TestUpdateMutations_AddField
2023/09/01 13:42:17 In TestUpdateMutations_AddField()
2023/09/01 13:42:17 Setting JSON docs in KV
2023/09/01 13:42:24 Created the secondary index index_newField. Waiting for it become active
2023/09/01 13:42:24 Index is 4749002792461696648 now active
2023/09/01 13:42:24 Using n1ql client
2023/09/01 13:42:24 Count of scan results before add field mutations:  0
2023/09/01 13:42:24 Expected and Actual scan responses are the same
2023/09/01 13:42:24 Using n1ql client
2023/09/01 13:42:24 Count of scan results after add field mutations:  300
2023/09/01 13:42:24 Expected and Actual scan responses are the same
--- PASS: TestUpdateMutations_AddField (7.54s)
=== RUN   TestUpdateMutations_DataTypeChange
2023/09/01 13:42:24 In TestUpdateMutations_DataTypeChange()
2023/09/01 13:42:24 Setting JSON docs in KV
2023/09/01 13:42:32 Created the secondary index index_isUserActive. Waiting for it become active
2023/09/01 13:42:32 Index is 1228959661568748888 now active
2023/09/01 13:42:32 Using n1ql client
2023/09/01 13:42:32 Expected and Actual scan responses are the same
2023/09/01 13:42:33 Using n1ql client
2023/09/01 13:42:33 Expected and Actual scan responses are the same
2023/09/01 13:42:33 Using n1ql client
2023/09/01 13:42:33 Expected and Actual scan responses are the same
2023/09/01 13:42:33 Using n1ql client
2023/09/01 13:42:33 Expected and Actual scan responses are the same
--- PASS: TestUpdateMutations_DataTypeChange (8.74s)
=== RUN   TestMultipleBuckets
2023/09/01 13:42:33 In TestMultipleBuckets()
2023/09/01 13:42:33 In DropAllSecondaryIndexes()
2023/09/01 13:42:33 Index found:  index_isUserActive
2023/09/01 13:42:33 Dropped index index_isUserActive
2023/09/01 13:42:33 Index found:  index_bal
2023/09/01 13:42:33 Dropped index index_bal
2023/09/01 13:42:33 Index found:  index_LongPrimaryField
2023/09/01 13:42:33 Dropped index index_LongPrimaryField
2023/09/01 13:42:33 Index found:  index_newField
2023/09/01 13:42:33 Dropped index index_newField
2023/09/01 13:43:11 Flushed the bucket default, Response body: 
2023/09/01 13:43:14 Modified parameters of bucket default, responseBody: 
2023/09/01 13:43:14 Created bucket testbucket2, responseBody: 
2023/09/01 13:43:14 Created bucket testbucket3, responseBody: 
2023/09/01 13:43:14 Created bucket testbucket4, responseBody: 
2023/09/01 13:43:29 Generating docs and Populating all the buckets
2023/09/01 13:43:33 Created the secondary index bucket1_age. Waiting for it become active
2023/09/01 13:43:33 Index is 2696123838481020225 now active
2023/09/01 13:43:39 Created the secondary index bucket2_city. Waiting for it become active
2023/09/01 13:43:39 Index is 13003362419305011588 now active
2023/09/01 13:43:46 Created the secondary index bucket3_gender. Waiting for it become active
2023/09/01 13:43:46 Index is 467995392781020784 now active
2023/09/01 13:43:52 Created the secondary index bucket4_balance. Waiting for it become active
2023/09/01 13:43:52 Index is 4993122022002285932 now active
2023/09/01 13:43:55 Using n1ql client
2023/09/01 13:43:55 Expected and Actual scan responses are the same
2023/09/01 13:43:55 Using n1ql client
2023-09-01T13:43:55.885+05:30 [Info] GSIC[default/testbucket2-_default-_default-1693556035881890498] started ...
2023/09/01 13:43:55 Expected and Actual scan responses are the same
2023/09/01 13:43:55 Using n1ql client
2023-09-01T13:43:55.897+05:30 [Info] GSIC[default/testbucket3-_default-_default-1693556035894082613] started ...
2023/09/01 13:43:55 Expected and Actual scan responses are the same
2023/09/01 13:43:55 Using n1ql client
2023-09-01T13:43:55.907+05:30 [Info] GSIC[default/testbucket4-_default-_default-1693556035904991593] started ...
2023/09/01 13:43:55 Expected and Actual scan responses are the same
2023/09/01 13:43:58 Deleted bucket testbucket2, responseBody: 
2023/09/01 13:44:00 Deleted bucket testbucket3, responseBody: 
2023/09/01 13:44:02 Deleted bucket testbucket4, responseBody: 
2023/09/01 13:44:05 Modified parameters of bucket default, responseBody: 
--- PASS: TestMultipleBuckets (107.64s)
=== RUN   TestBucketFlush
2023/09/01 13:44:20 In TestBucketFlush()
2023/09/01 13:44:20 In DropAllSecondaryIndexes()
2023/09/01 13:44:20 Index found:  bucket1_age
2023/09/01 13:44:21 Dropped index bucket1_age
2023/09/01 13:44:59 Flushed the bucket default, Response body: 
2023/09/01 13:45:03 Created the secondary index index_age. Waiting for it become active
2023/09/01 13:45:03 Index is 8298838699853720621 now active
2023/09/01 13:45:03 Using n1ql client
2023/09/01 13:45:04 Expected and Actual scan responses are the same
2023/09/01 13:45:10 Created the secondary index index_gender. Waiting for it become active
2023/09/01 13:45:10 Index is 943185124555212378 now active
2023/09/01 13:45:10 Using n1ql client
2023/09/01 13:45:11 Expected and Actual scan responses are the same
2023/09/01 13:45:16 Created the secondary index index_city. Waiting for it become active
2023/09/01 13:45:16 Index is 17864269847577780883 now active
2023/09/01 13:45:16 Using n1ql client
2023/09/01 13:45:16 Expected and Actual scan responses are the same
2023/09/01 13:45:54 Flushed the bucket default, Response body: 
2023/09/01 13:45:54 TestBucketFlush:: Flushed the bucket
2023/09/01 13:45:54 Using n1ql client
2023/09/01 13:45:54 Using n1ql client
2023/09/01 13:45:54 Using n1ql client
--- PASS: TestBucketFlush (93.69s)
=== RUN   TestLargeDocumentSize
2023/09/01 13:45:54 In TestLargeDocumentSize()
2023/09/01 13:45:54 Data file exists. Skipping download
2023/09/01 13:45:54 Length of docs and largeDocs = 200 and 200
2023/09/01 13:45:59 Created the secondary index index_userscreenname. Waiting for it become active
2023/09/01 13:45:59 Index is 4877455884733926259 now active
2023/09/01 13:45:59 Using n1ql client
2023/09/01 13:45:59 Expected and Actual scan responses are the same
--- PASS: TestLargeDocumentSize (4.46s)
=== RUN   TestFieldsWithSpecialCharacters
2023/09/01 13:45:59 In TestFieldsWithSpecialCharacters()
2023/09/01 13:46:05 Created the secondary index index_specialchar. Waiting for it become active
2023/09/01 13:46:05 Index is 18378716266909655441 now active
2023/09/01 13:46:05 Looking up for value ,†šwv¶ß
2023/09/01 13:46:05 Using n1ql client
2023/09/01 13:46:06 Expected and Actual scan responses are the same
--- PASS: TestFieldsWithSpecialCharacters (7.49s)
=== RUN   TestLargeKeyLookup
2023/09/01 13:46:06 In TestLargeKeyLookup()
2023/09/01 13:46:13 Created the secondary index index_largeKeyLookup. Waiting for it become active
2023/09/01 13:46:13 Index is 335197415642429122 now active
2023/09/01 13:46:13 Looking up for a large key
2023/09/01 13:46:13 Using n1ql client
2023/09/01 13:46:14 Expected and Actual scan responses are the same
--- PASS: TestLargeKeyLookup (8.00s)
=== RUN   TestIndexNameValidation
2023/09/01 13:46:14 In TestIndexNameValidation()
2023/09/01 13:46:14 Setting JSON docs in KV
2023/09/01 13:46:15 Creation of index with invalid name ÌñÐÉx&(abc_% failed as expected
2023/09/01 13:46:20 Created the secondary index #primary-Index_test. Waiting for it become active
2023/09/01 13:46:20 Index is 18335239097172370589 now active
2023/09/01 13:46:20 Using n1ql client
2023/09/01 13:46:20 Expected and Actual scan responses are the same
--- PASS: TestIndexNameValidation (5.77s)
=== RUN   TestSameFieldNameAtDifferentLevels
2023/09/01 13:46:20 In TestSameFieldNameAtDifferentLevels()
2023/09/01 13:46:20 Setting JSON docs in KV
2023/09/01 13:46:27 Created the secondary index cityindex. Waiting for it become active
2023/09/01 13:46:27 Index is 228373084376623576 now active
2023/09/01 13:46:27 Using n1ql client
2023/09/01 13:46:27 Expected and Actual scan responses are the same
--- PASS: TestSameFieldNameAtDifferentLevels (7.14s)
=== RUN   TestSameIndexNameInTwoBuckets
2023/09/01 13:46:27 In TestSameIndexNameInTwoBuckets()
2023/09/01 13:46:27 In DropAllSecondaryIndexes()
2023/09/01 13:46:27 Index found:  #primary-Index_test
2023/09/01 13:46:27 Dropped index #primary-Index_test
2023/09/01 13:46:27 Index found:  index_specialchar
2023/09/01 13:46:27 Dropped index index_specialchar
2023/09/01 13:46:27 Index found:  index_gender
2023/09/01 13:46:27 Dropped index index_gender
2023/09/01 13:46:27 Index found:  index_userscreenname
2023/09/01 13:46:27 Dropped index index_userscreenname
2023/09/01 13:46:27 Index found:  index_largeKeyLookup
2023/09/01 13:46:27 Dropped index index_largeKeyLookup
2023/09/01 13:46:27 Index found:  index_age
2023/09/01 13:46:28 Dropped index index_age
2023/09/01 13:46:28 Index found:  cityindex
2023/09/01 13:46:28 Dropped index cityindex
2023/09/01 13:46:28 Index found:  index_city
2023/09/01 13:46:28 Dropped index index_city
2023/09/01 13:47:06 Flushed the bucket default, Response body: 
2023/09/01 13:47:09 Modified parameters of bucket default, responseBody: 
2023/09/01 13:47:09 Created bucket buck2, responseBody: 
2023/09/01 13:47:24 Generating docs and Populating all the buckets
2023/09/01 13:47:28 Created the secondary index b_idx. Waiting for it become active
2023/09/01 13:47:28 Index is 14316829411227498991 now active
2023/09/01 13:47:34 Created the secondary index b_idx. Waiting for it become active
2023/09/01 13:47:34 Index is 14136532910281016164 now active
2023/09/01 13:47:37 Using n1ql client
2023/09/01 13:47:37 Expected and Actual scan responses are the same
2023/09/01 13:47:37 Using n1ql client
2023-09-01T13:47:37.891+05:30 [Info] GSIC[default/buck2-_default-_default-1693556257888231700] started ...
2023/09/01 13:47:37 Expected and Actual scan responses are the same
2023/09/01 13:47:40 Modified parameters of bucket default, responseBody: 
2023/09/01 13:47:43 Deleted bucket buck2, responseBody: 
--- PASS: TestSameIndexNameInTwoBuckets (90.77s)
=== RUN   TestLargeKeysSplChars
2023/09/01 13:47:58 In TestLargeKeysSplChars()
2023/09/01 13:48:08 Created the secondary index idspl1. Waiting for it become active
2023/09/01 13:48:08 Index is 1733576294355277977 now active
2023/09/01 13:48:17 Created the secondary index idspl2. Waiting for it become active
2023/09/01 13:48:17 Index is 16015741178517556208 now active
2023/09/01 13:48:24 Created the secondary index idspl3. Waiting for it become active
2023/09/01 13:48:24 Index is 5344114053459978213 now active
2023/09/01 13:48:24 Using n1ql client
2023/09/01 13:48:24 Expected and Actual scan responses are the same
2023-09-01T13:48:24.793+05:30 [Error] transport error between 127.0.0.1:55820->127.0.0.1:9107: write tcp 127.0.0.1:55820->127.0.0.1:9107: write: broken pipe
2023-09-01T13:48:24.793+05:30 [Error] [GsiScanClient:"127.0.0.1:9107"]  request transport failed `write tcp 127.0.0.1:55820->127.0.0.1:9107: write: broken pipe`
2023-09-01T13:48:24.794+05:30 [Error] metadataClient:PickRandom: Fail to find indexer for all index partitions. Num partition 1.  Partition with instances 0 
2023-09-01T13:48:24.794+05:30 [Error] metadataClient:PickRandom: Replicas - [15749662856627824677], PrunedReplica - map[], FilteredReplica map[]
2023/09/01 13:48:25 Expected and Actual scan responses are the same
2023/09/01 13:48:25 Using n1ql client
2023/09/01 13:48:25 Expected and Actual scan responses are the same
--- PASS: TestLargeKeysSplChars (26.95s)
=== RUN   TestVeryLargeIndexKey
2023/09/01 13:48:25 In DropAllSecondaryIndexes()
2023/09/01 13:48:25 Index found:  idspl2
2023/09/01 13:48:25 Dropped index idspl2
2023/09/01 13:48:25 Index found:  idspl3
2023/09/01 13:48:25 Dropped index idspl3
2023/09/01 13:48:25 Index found:  idspl1
2023/09/01 13:48:25 Dropped index idspl1
2023/09/01 13:48:25 Index found:  b_idx
2023/09/01 13:48:25 Dropped index b_idx
2023/09/01 13:49:03 Flushed the bucket default, Response body: 
2023/09/01 13:49:03 TestVeryLargeIndexKey:: Flushed the bucket
2023/09/01 13:49:04 clusterconfig.KVAddress = 127.0.0.1:9000
2023/09/01 13:49:09 Created the secondary index i1. Waiting for it become active
2023/09/01 13:49:09 Index is 3852577247658300381 now active
2023/09/01 13:49:09 Using n1ql client
2023/09/01 13:49:09 Expected and Actual scan responses are the same
2023/09/01 13:49:16 Created the secondary index i2. Waiting for it become active
2023/09/01 13:49:16 Index is 6063915149215592097 now active
2023/09/01 13:49:16 Using n1ql client
2023/09/01 13:49:18 Expected and Actual scan responses are the same
2023/09/01 13:49:18 In DropAllSecondaryIndexes()
2023/09/01 13:49:18 Index found:  i1
2023/09/01 13:49:18 Dropped index i1
2023/09/01 13:49:18 Index found:  i2
2023/09/01 13:49:18 Dropped index i2
2023/09/01 13:49:56 Flushed the bucket default, Response body: 
--- PASS: TestVeryLargeIndexKey (91.23s)
=== RUN   TestTempBufScanResult
2023/09/01 13:49:56 In DropAllSecondaryIndexes()
2023/09/01 13:50:34 Flushed the bucket default, Response body: 
2023/09/01 13:50:34 TestTempBufScanResult:: Flushed the bucket
2023/09/01 13:50:37 Created the secondary index index_idxKey. Waiting for it become active
2023/09/01 13:50:37 Index is 14787317968670169458 now active
2023/09/01 13:50:37 Using n1ql client
2023/09/01 13:50:37 Expected and Actual scan responses are the same
2023/09/01 13:50:37 In DropAllSecondaryIndexes()
2023/09/01 13:50:37 Index found:  index_idxKey
2023/09/01 13:50:37 Dropped index index_idxKey
2023/09/01 13:51:15 Flushed the bucket default, Response body: 
--- PASS: TestTempBufScanResult (79.25s)
=== RUN   TestBuildDeferredAnotherBuilding
2023/09/01 13:51:15 In TestBuildDeferredAnotherBuilding()
2023/09/01 13:51:15 In DropAllSecondaryIndexes()
2023/09/01 13:51:59 Setting JSON docs in KV
2023/09/01 13:53:55 Build the deferred index id_age1. Waiting for the index to become active
2023/09/01 13:53:55 Waiting for index 5350437066089297541 to go active ...
2023/09/01 13:53:56 Waiting for index 5350437066089297541 to go active ...
2023/09/01 13:53:57 Waiting for index 5350437066089297541 to go active ...
2023/09/01 13:53:58 Waiting for index 5350437066089297541 to go active ...
2023/09/01 13:53:59 Waiting for index 5350437066089297541 to go active ...
2023/09/01 13:54:00 Waiting for index 5350437066089297541 to go active ...
2023/09/01 13:54:01 Waiting for index 5350437066089297541 to go active ...
2023/09/01 13:54:02 Waiting for index 5350437066089297541 to go active ...
2023/09/01 13:54:03 Waiting for index 5350437066089297541 to go active ...
2023/09/01 13:54:04 Waiting for index 5350437066089297541 to go active ...
2023/09/01 13:54:05 Waiting for index 5350437066089297541 to go active ...
2023/09/01 13:54:06 Waiting for index 5350437066089297541 to go active ...
2023/09/01 13:54:07 Waiting for index 5350437066089297541 to go active ...
2023/09/01 13:54:08 Waiting for index 5350437066089297541 to go active ...
2023/09/01 13:54:09 Waiting for index 5350437066089297541 to go active ...
2023/09/01 13:54:10 Waiting for index 5350437066089297541 to go active ...
2023/09/01 13:54:11 Waiting for index 5350437066089297541 to go active ...
2023/09/01 13:54:12 Waiting for index 5350437066089297541 to go active ...
2023/09/01 13:54:13 Waiting for index 5350437066089297541 to go active ...
2023/09/01 13:54:14 Waiting for index 5350437066089297541 to go active ...
2023/09/01 13:54:15 Index is 5350437066089297541 now active
2023/09/01 13:54:15 Build command issued for the deferred indexes [3804504366467420015]
2023/09/01 13:54:18 Build index failed as expected: Build index failed. Encountered transient error. Index id_age will retry building in the background for reason: Build Already In Progress. Keyspace default.
2023/09/01 13:54:18 Waiting for index 3804504366467420015 to go active ...
2023/09/01 13:54:19 Waiting for index 3804504366467420015 to go active ...
2023/09/01 13:54:20 Waiting for index 3804504366467420015 to go active ...
2023/09/01 13:54:21 Waiting for index 3804504366467420015 to go active ...
2023/09/01 13:54:22 Waiting for index 3804504366467420015 to go active ...
2023/09/01 13:54:23 Waiting for index 3804504366467420015 to go active ...
2023/09/01 13:54:24 Waiting for index 3804504366467420015 to go active ...
2023/09/01 13:54:25 Waiting for index 3804504366467420015 to go active ...
2023/09/01 13:54:26 Waiting for index 3804504366467420015 to go active ...
2023/09/01 13:54:27 Waiting for index 3804504366467420015 to go active ...
2023/09/01 13:54:28 Waiting for index 3804504366467420015 to go active ...
2023/09/01 13:54:29 Waiting for index 3804504366467420015 to go active ...
2023/09/01 13:54:30 Waiting for index 3804504366467420015 to go active ...
2023/09/01 13:54:31 Waiting for index 3804504366467420015 to go active ...
2023/09/01 13:54:32 Waiting for index 3804504366467420015 to go active ...
2023/09/01 13:54:33 Waiting for index 3804504366467420015 to go active ...
2023/09/01 13:54:34 Waiting for index 3804504366467420015 to go active ...
2023/09/01 13:54:35 Waiting for index 3804504366467420015 to go active ...
2023/09/01 13:54:36 Index is 3804504366467420015 now active
2023/09/01 13:54:36 Waiting for index 7610251343674835170 to go active ...
2023/09/01 13:54:37 Waiting for index 7610251343674835170 to go active ...
2023/09/01 13:54:38 Waiting for index 7610251343674835170 to go active ...
2023/09/01 13:54:39 Waiting for index 7610251343674835170 to go active ...
2023/09/01 13:54:40 Waiting for index 7610251343674835170 to go active ...
2023/09/01 13:54:41 Waiting for index 7610251343674835170 to go active ...
2023/09/01 13:54:42 Waiting for index 7610251343674835170 to go active ...
2023/09/01 13:54:43 Waiting for index 7610251343674835170 to go active ...
2023/09/01 13:54:44 Waiting for index 7610251343674835170 to go active ...
2023/09/01 13:54:45 Waiting for index 7610251343674835170 to go active ...
2023/09/01 13:54:46 Waiting for index 7610251343674835170 to go active ...
2023/09/01 13:54:47 Waiting for index 7610251343674835170 to go active ...
2023/09/01 13:54:48 Waiting for index 7610251343674835170 to go active ...
2023/09/01 13:54:49 Waiting for index 7610251343674835170 to go active ...
2023/09/01 13:54:50 Waiting for index 7610251343674835170 to go active ...
2023/09/01 13:54:51 Waiting for index 7610251343674835170 to go active ...
2023/09/01 13:54:52 Waiting for index 7610251343674835170 to go active ...
2023/09/01 13:54:53 Waiting for index 7610251343674835170 to go active ...
2023/09/01 13:54:54 Index is 7610251343674835170 now active
2023/09/01 13:54:54 Using n1ql client
2023/09/01 13:54:54 Expected and Actual scan responses are the same
2023/09/01 13:54:54 Using n1ql client
2023/09/01 13:54:55 Expected and Actual scan responses are the same
--- PASS: TestBuildDeferredAnotherBuilding (219.66s)
=== RUN   TestMultipleBucketsDeferredBuild
2023/09/01 13:54:55 In TestMultipleBucketsDeferredBuild()
2023/09/01 13:54:55 In DropAllSecondaryIndexes()
2023/09/01 13:54:55 Index found:  id_age1
2023/09/01 13:54:55 Dropped index id_age1
2023/09/01 13:54:55 Index found:  id_age
2023/09/01 13:54:55 Dropped index id_age
2023/09/01 13:54:55 Index found:  id_company
2023/09/01 13:54:55 Dropped index id_company
2023/09/01 13:55:32 Flushed the bucket default, Response body: 
2023/09/01 13:55:35 Modified parameters of bucket default, responseBody: 
2023/09/01 13:55:35 http://127.0.0.1:9000/pools/default/buckets/defertest_buck2
2023/09/01 13:55:35 &{DELETE http://127.0.0.1:9000/pools/default/buckets/defertest_buck2 HTTP/1.1 1 1 map[Authorization:[Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=] Content-Type:[application/x-www-form-urlencoded; charset=UTF-8]]   0 [] false 127.0.0.1:9000 map[] map[]  map[]      0xc00012e000}
2023/09/01 13:55:35 &{404 Object Not Found 404 HTTP/1.1 1 1 map[Cache-Control:[no-cache,no-store,must-revalidate] Content-Length:[31] Content-Type:[text/plain] Date:[Fri, 01 Sep 2023 08:25:35 GMT] Expires:[Thu, 01 Jan 1970 00:00:00 GMT] Pragma:[no-cache] Server:[Couchbase Server] X-Content-Type-Options:[nosniff] X-Frame-Options:[DENY] X-Permitted-Cross-Domain-Policies:[none] X-Xss-Protection:[1; mode=block]] 0xc048e34280 31 [] false false map[] 0xc000454000 }
2023/09/01 13:55:35 DeleteBucket failed for bucket defertest_buck2 
2023/09/01 13:55:35 Deleted bucket defertest_buck2, responseBody: Requested resource not found.
2023/09/01 13:55:35 Created bucket defertest_buck2, responseBody: 
2023/09/01 13:55:50 Setting JSON docs in KV
2023/09/01 13:57:05 Build command issued for the deferred indexes [14554880784757601948]
2023/09/01 13:57:06 Build command issued for the deferred indexes [9401059743343697471 8484598016194388767]
2023/09/01 13:57:06 Index state of 8484598016194388767 is INDEX_STATE_READY
2023/09/01 13:57:06 Waiting for index 14554880784757601948 to go active ...
2023/09/01 13:57:07 Waiting for index 14554880784757601948 to go active ...
2023/09/01 13:57:08 Waiting for index 14554880784757601948 to go active ...
2023/09/01 13:57:09 Waiting for index 14554880784757601948 to go active ...
2023/09/01 13:57:10 Waiting for index 14554880784757601948 to go active ...
2023/09/01 13:57:11 Waiting for index 14554880784757601948 to go active ...
2023/09/01 13:57:12 Waiting for index 14554880784757601948 to go active ...
2023/09/01 13:57:13 Waiting for index 14554880784757601948 to go active ...
2023/09/01 13:57:14 Waiting for index 14554880784757601948 to go active ...
2023/09/01 13:57:15 Waiting for index 14554880784757601948 to go active ...
2023/09/01 13:57:16 Waiting for index 14554880784757601948 to go active ...
2023/09/01 13:57:17 Waiting for index 14554880784757601948 to go active ...
2023/09/01 13:57:18 Index is 14554880784757601948 now active
2023/09/01 13:57:18 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:19 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:20 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:21 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:22 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:23 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:24 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:25 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:26 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:27 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:28 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:29 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:30 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:31 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:32 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:33 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:34 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:35 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:36 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:37 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:38 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:39 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:40 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:41 Waiting for index 9401059743343697471 to go active ...
2023/09/01 13:57:42 Index is 9401059743343697471 now active
2023/09/01 13:57:42 Using n1ql client
2023/09/01 13:57:42 Expected and Actual scan responses are the same
2023/09/01 13:57:42 Using n1ql client
2023/09/01 13:57:42 Expected and Actual scan responses are the same
2023/09/01 13:57:42 Using n1ql client
2023-09-01T13:57:42.541+05:30 [Info] GSIC[default/defertest_buck2-_default-_default-1693556862538617276] started ...
2023/09/01 13:57:42 Expected and Actual scan responses are the same
2023/09/01 13:57:42 http://127.0.0.1:9000/pools/default/buckets/default
2023/09/01 13:57:42 &{POST http://127.0.0.1:9000/pools/default/buckets/default HTTP/1.1 1 1 map[Authorization:[Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=] Content-Type:[application/x-www-form-urlencoded; charset=UTF-8]] {0xc0222bb960} 0x77d9e0 28 [] false 127.0.0.1:9000 map[] map[]  map[]      0xc00012e000}
2023/09/01 13:57:42 &{400 Bad Request 400 HTTP/1.1 1 1 map[Cache-Control:[no-cache,no-store,must-revalidate] Content-Length:[388] Content-Type:[application/json] Date:[Fri, 01 Sep 2023 08:27:42 GMT] Expires:[Thu, 01 Jan 1970 00:00:00 GMT] Pragma:[no-cache] Server:[Couchbase Server] X-Content-Type-Options:[nosniff] X-Frame-Options:[DENY] X-Permitted-Cross-Domain-Policies:[none] X-Xss-Protection:[1; mode=block]] 0xc0069ee000 388 [] false false map[] 0xc00437fd00 }
2023/09/01 13:57:42 EditBucket failed for bucket default 
2023/09/01 13:57:45 Modified parameters of bucket default, responseBody: {"errors":{"ramQuota":"RAM quota specified is too large to be provisioned into this cluster."},"summaries":{"ramSummary":{"total":3145728000,"otherBuckets":1073741824,"nodesCount":2,"perNodeMegs":1024,"thisAlloc":2147483648,"thisUsed":134856656,"free":-75497472},"hddSummary":{"total":78981824512,"otherData":56525342064,"otherBuckets":191098880,"thisUsed":150472704,"free":22265383568}}}
2023/09/01 13:57:47 Deleted bucket defertest_buck2, responseBody: 
--- PASS: TestMultipleBucketsDeferredBuild (177.42s)
=== RUN   TestCreateDropCreateDeferredIndex
2023/09/01 13:57:52 In TestCreateDropCreateDeferredIndex()
2023/09/01 13:57:52 In DropAllSecondaryIndexes()
2023/09/01 13:57:52 Index found:  buck1_id2
2023/09/01 13:57:52 Dropped index buck1_id2
2023/09/01 13:57:52 Index found:  buck1_id1
2023/09/01 13:57:52 Dropped index buck1_id1
2023/09/01 13:57:55 Setting JSON docs in KV
2023/09/01 13:58:10 Created the secondary index id_company. Waiting for it become active
2023/09/01 13:58:10 Index is 15626316330156728772 now active
2023/09/01 13:58:11 Dropping the secondary index id_age
2023/09/01 13:58:11 Index dropped
2023/09/01 13:58:14 Setting JSON docs in KV
2023/09/01 13:58:23 Using n1ql client
2023/09/01 13:58:23 Expected and Actual scan responses are the same
--- PASS: TestCreateDropCreateDeferredIndex (30.50s)
=== RUN   TestMultipleDeferredIndexes_BuildTogether
2023/09/01 13:58:23 In TestMultipleDeferredIndexes_BuildTogether()
2023/09/01 13:58:23 In DropAllSecondaryIndexes()
2023/09/01 13:58:23 Index found:  id_company
2023/09/01 13:58:23 Dropped index id_company
2023/09/01 13:58:26 Setting JSON docs in KV
2023/09/01 13:58:39 Created the secondary index id_company. Waiting for it become active
2023/09/01 13:58:39 Index is 1559387099719222411 now active
2023/09/01 13:58:41 Build command issued for the deferred indexes [id_age id_gender id_isActive], bucket: default, scope: _default, coll: _default
2023/09/01 13:58:41 Waiting for the index id_age to become active
2023/09/01 13:58:41 Waiting for index 18391780063008290031 to go active ...
2023/09/01 13:58:42 Waiting for index 18391780063008290031 to go active ...
2023/09/01 13:58:43 Waiting for index 18391780063008290031 to go active ...
2023/09/01 13:58:44 Waiting for index 18391780063008290031 to go active ...
2023/09/01 13:58:45 Waiting for index 18391780063008290031 to go active ...
2023/09/01 13:58:46 Waiting for index 18391780063008290031 to go active ...
2023/09/01 13:58:47 Waiting for index 18391780063008290031 to go active ...
2023/09/01 13:58:48 Waiting for index 18391780063008290031 to go active ...
2023/09/01 13:58:49 Waiting for index 18391780063008290031 to go active ...
2023/09/01 13:58:50 Waiting for index 18391780063008290031 to go active ...
2023/09/01 13:58:51 Waiting for index 18391780063008290031 to go active ...
2023/09/01 13:58:52 Waiting for index 18391780063008290031 to go active ...
2023/09/01 13:58:53 Waiting for index 18391780063008290031 to go active ...
2023/09/01 13:58:54 Waiting for index 18391780063008290031 to go active ...
2023/09/01 13:58:55 Waiting for index 18391780063008290031 to go active ...
2023/09/01 13:58:56 Waiting for index 18391780063008290031 to go active ...
2023/09/01 13:58:57 Waiting for index 18391780063008290031 to go active ...
2023/09/01 13:58:58 Waiting for index 18391780063008290031 to go active ...
2023/09/01 13:58:59 Waiting for index 18391780063008290031 to go active ...
2023/09/01 13:59:00 Index is 18391780063008290031 now active
2023/09/01 13:59:00 Waiting for the index id_gender to become active
2023/09/01 13:59:00 Index is 438957766381664598 now active
2023/09/01 13:59:00 Waiting for the index id_isActive to become active
2023/09/01 13:59:00 Index is 9454606934086602365 now active
2023/09/01 13:59:00 Using n1ql client
2023/09/01 13:59:00 Expected and Actual scan responses are the same
2023/09/01 13:59:02 Setting JSON docs in KV
2023/09/01 13:59:13 Using n1ql client
2023/09/01 13:59:13 Expected and Actual scan responses are the same
2023/09/01 13:59:13 Using n1ql client
2023/09/01 13:59:13 Expected and Actual scan responses are the same
--- PASS: TestMultipleDeferredIndexes_BuildTogether (50.64s)
=== RUN   TestMultipleDeferredIndexes_BuildOneByOne
2023/09/01 13:59:13 In TestMultipleDeferredIndexes_BuildOneByOne()
2023/09/01 13:59:13 In DropAllSecondaryIndexes()
2023/09/01 13:59:13 Index found:  id_company
2023/09/01 13:59:14 Dropped index id_company
2023/09/01 13:59:14 Index found:  id_age
2023/09/01 13:59:14 Dropped index id_age
2023/09/01 13:59:14 Index found:  id_gender
2023/09/01 13:59:14 Dropped index id_gender
2023/09/01 13:59:14 Index found:  id_isActive
2023/09/01 13:59:14 Dropped index id_isActive
2023/09/01 13:59:16 Setting JSON docs in KV
2023/09/01 13:59:32 Created the secondary index id_company. Waiting for it become active
2023/09/01 13:59:32 Index is 223527561583430434 now active
2023/09/01 13:59:33 Build command issued for the deferred indexes [id_age], bucket: default, scope: _default, coll: _default
2023/09/01 13:59:33 Waiting for the index id_age to become active
2023/09/01 13:59:33 Waiting for index 6386163684634377854 to go active ...
2023/09/01 13:59:34 Waiting for index 6386163684634377854 to go active ...
2023/09/01 13:59:35 Waiting for index 6386163684634377854 to go active ...
2023/09/01 13:59:36 Waiting for index 6386163684634377854 to go active ...
2023/09/01 13:59:37 Waiting for index 6386163684634377854 to go active ...
2023/09/01 13:59:38 Waiting for index 6386163684634377854 to go active ...
2023/09/01 13:59:39 Waiting for index 6386163684634377854 to go active ...
2023/09/01 13:59:40 Waiting for index 6386163684634377854 to go active ...
2023/09/01 13:59:41 Waiting for index 6386163684634377854 to go active ...
2023/09/01 13:59:42 Waiting for index 6386163684634377854 to go active ...
2023/09/01 13:59:43 Waiting for index 6386163684634377854 to go active ...
2023/09/01 13:59:44 Waiting for index 6386163684634377854 to go active ...
2023/09/01 13:59:45 Index is 6386163684634377854 now active
2023/09/01 13:59:45 Build command issued for the deferred indexes [id_gender], bucket: default, scope: _default, coll: _default
2023/09/01 13:59:45 Waiting for the index id_gender to become active
2023/09/01 13:59:45 Waiting for index 5989468454770832854 to go active ...
2023/09/01 13:59:46 Waiting for index 5989468454770832854 to go active ...
2023/09/01 13:59:47 Waiting for index 5989468454770832854 to go active ...
2023/09/01 13:59:48 Waiting for index 5989468454770832854 to go active ...
2023/09/01 13:59:49 Waiting for index 5989468454770832854 to go active ...
2023/09/01 13:59:50 Waiting for index 5989468454770832854 to go active ...
2023/09/01 13:59:51 Waiting for index 5989468454770832854 to go active ...
2023/09/01 13:59:52 Waiting for index 5989468454770832854 to go active ...
2023/09/01 13:59:53 Waiting for index 5989468454770832854 to go active ...
2023/09/01 13:59:54 Waiting for index 5989468454770832854 to go active ...
2023/09/01 13:59:55 Waiting for index 5989468454770832854 to go active ...
2023/09/01 13:59:56 Waiting for index 5989468454770832854 to go active ...
2023/09/01 13:59:57 Index is 5989468454770832854 now active
2023/09/01 13:59:57 Build command issued for the deferred indexes [id_isActive], bucket: default, scope: _default, coll: _default
2023/09/01 13:59:57 Waiting for the index id_isActive to become active
2023/09/01 13:59:57 Waiting for index 14588380570817552130 to go active ...
2023/09/01 13:59:58 Waiting for index 14588380570817552130 to go active ...
2023/09/01 13:59:59 Waiting for index 14588380570817552130 to go active ...
2023/09/01 14:00:00 Waiting for index 14588380570817552130 to go active ...
2023/09/01 14:00:01 Waiting for index 14588380570817552130 to go active ...
2023/09/01 14:00:02 Waiting for index 14588380570817552130 to go active ...
2023/09/01 14:00:03 Waiting for index 14588380570817552130 to go active ...
2023/09/01 14:00:04 Waiting for index 14588380570817552130 to go active ...
2023/09/01 14:00:05 Waiting for index 14588380570817552130 to go active ...
2023/09/01 14:00:06 Waiting for index 14588380570817552130 to go active ...
2023/09/01 14:00:07 Waiting for index 14588380570817552130 to go active ...
2023/09/01 14:00:08 Waiting for index 14588380570817552130 to go active ...
2023/09/01 14:00:09 Waiting for index 14588380570817552130 to go active ...
2023/09/01 14:00:10 Index is 14588380570817552130 now active
2023/09/01 14:00:11 Using n1ql client
2023/09/01 14:00:11 Expected and Actual scan responses are the same
2023/09/01 14:00:13 Setting JSON docs in KV
2023/09/01 14:00:23 Using n1ql client
2023/09/01 14:00:23 Expected and Actual scan responses are the same
2023/09/01 14:00:23 Using n1ql client
2023/09/01 14:00:24 Expected and Actual scan responses are the same
--- PASS: TestMultipleDeferredIndexes_BuildOneByOne (70.25s)
=== RUN   TestDropDeferredIndexWhileOthersBuilding
2023/09/01 14:00:24 In TestDropDeferredIndexWhileOthersBuilding()
2023/09/01 14:00:24 In DropAllSecondaryIndexes()
2023/09/01 14:00:24 Index found:  id_age
2023/09/01 14:00:24 Dropped index id_age
2023/09/01 14:00:24 Index found:  id_company
2023/09/01 14:00:24 Dropped index id_company
2023/09/01 14:00:24 Index found:  id_gender
2023/09/01 14:00:24 Dropped index id_gender
2023/09/01 14:00:24 Index found:  id_isActive
2023/09/01 14:00:24 Dropped index id_isActive
2023/09/01 14:00:27 Setting JSON docs in KV
2023/09/01 14:00:43 Created the secondary index id_company. Waiting for it become active
2023/09/01 14:00:43 Index is 10869023717887777807 now active
2023/09/01 14:00:45 Build command issued for the deferred indexes [5250360486191716615 16363136979054125262]
2023/09/01 14:00:47 Dropping the secondary index id_isActive
2023/09/01 14:00:47 Index dropped
2023/09/01 14:00:47 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:00:48 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:00:49 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:00:50 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:00:51 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:00:52 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:00:53 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:00:54 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:00:55 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:00:56 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:00:57 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:00:58 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:00:59 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:01:00 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:01:01 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:01:02 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:01:03 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:01:04 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:01:05 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:01:06 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:01:07 Waiting for index 5250360486191716615 to go active ...
2023/09/01 14:01:08 Index is 5250360486191716615 now active
2023/09/01 14:01:08 Index is 16363136979054125262 now active
2023/09/01 14:01:09 Using n1ql client
2023/09/01 14:01:09 Expected and Actual scan responses are the same
2023/09/01 14:01:09 Using n1ql client
2023/09/01 14:01:09 Expected and Actual scan responses are the same
2023/09/01 14:01:12 Setting JSON docs in KV
2023/09/01 14:01:21 Using n1ql client
2023/09/01 14:01:21 Expected and Actual scan responses are the same
--- PASS: TestDropDeferredIndexWhileOthersBuilding (57.80s)
=== RUN   TestDropBuildingDeferredIndex
2023/09/01 14:01:21 In TestDropBuildingDeferredIndex()
2023/09/01 14:01:21 In DropAllSecondaryIndexes()
2023/09/01 14:01:21 Index found:  id_age
2023/09/01 14:01:22 Dropped index id_age
2023/09/01 14:01:22 Index found:  id_gender
2023/09/01 14:01:22 Dropped index id_gender
2023/09/01 14:01:22 Index found:  id_company
2023/09/01 14:01:22 Dropped index id_company
2023/09/01 14:01:24 Setting JSON docs in KV
2023/09/01 14:01:30 Build command issued for the deferred indexes [8401109373287203382 5679176057871214875]
2023/09/01 14:01:31 Dropping the secondary index id_age
2023/09/01 14:01:31 Index dropped
2023/09/01 14:01:31 Waiting for index 8401109373287203382 to go active ...
2023/09/01 14:01:32 Waiting for index 8401109373287203382 to go active ...
2023/09/01 14:01:33 Waiting for index 8401109373287203382 to go active ...
2023/09/01 14:01:34 Waiting for index 8401109373287203382 to go active ...
2023/09/01 14:01:35 Waiting for index 8401109373287203382 to go active ...
2023/09/01 14:01:36 Waiting for index 8401109373287203382 to go active ...
2023/09/01 14:01:37 Waiting for index 8401109373287203382 to go active ...
2023/09/01 14:01:38 Waiting for index 8401109373287203382 to go active ...
2023/09/01 14:01:39 Waiting for index 8401109373287203382 to go active ...
2023/09/01 14:01:40 Waiting for index 8401109373287203382 to go active ...
2023/09/01 14:01:41 Waiting for index 8401109373287203382 to go active ...
2023/09/01 14:01:42 Waiting for index 8401109373287203382 to go active ...
2023/09/01 14:01:43 Waiting for index 8401109373287203382 to go active ...
2023/09/01 14:01:44 Index is 8401109373287203382 now active
2023/09/01 14:01:44 Build command issued for the deferred indexes [id_gender], bucket: default, scope: _default, coll: _default
2023/09/01 14:01:44 Waiting for the index id_gender to become active
2023/09/01 14:01:44 Waiting for index 13532744107594140744 to go active ...
2023/09/01 14:01:45 Waiting for index 13532744107594140744 to go active ...
2023/09/01 14:01:46 Waiting for index 13532744107594140744 to go active ...
2023/09/01 14:01:47 Waiting for index 13532744107594140744 to go active ...
2023/09/01 14:01:48 Waiting for index 13532744107594140744 to go active ...
2023/09/01 14:01:49 Waiting for index 13532744107594140744 to go active ...
2023/09/01 14:01:50 Waiting for index 13532744107594140744 to go active ...
2023/09/01 14:01:51 Waiting for index 13532744107594140744 to go active ...
2023/09/01 14:01:52 Waiting for index 13532744107594140744 to go active ...
2023/09/01 14:01:53 Waiting for index 13532744107594140744 to go active ...
2023/09/01 14:01:54 Waiting for index 13532744107594140744 to go active ...
2023/09/01 14:01:55 Waiting for index 13532744107594140744 to go active ...
2023/09/01 14:01:56 Waiting for index 13532744107594140744 to go active ...
2023/09/01 14:01:57 Waiting for index 13532744107594140744 to go active ...
2023/09/01 14:01:58 Waiting for index 13532744107594140744 to go active ...
2023/09/01 14:01:59 Waiting for index 13532744107594140744 to go active ...
2023/09/01 14:02:00 Index is 13532744107594140744 now active
2023/09/01 14:02:01 Using n1ql client
2023/09/01 14:02:01 Expected and Actual scan responses are the same
2023/09/01 14:02:02 Using n1ql client
2023/09/01 14:02:02 Expected and Actual scan responses are the same
2023/09/01 14:02:04 Setting JSON docs in KV
2023/09/01 14:02:14 Using n1ql client
2023/09/01 14:02:15 Expected and Actual scan responses are the same
--- PASS: TestDropBuildingDeferredIndex (53.31s)
=== RUN   TestDropMultipleBuildingDeferredIndexes
2023/09/01 14:02:15 In TestDropMultipleBuildingDeferredIndexes()
2023/09/01 14:02:15 In DropAllSecondaryIndexes()
2023/09/01 14:02:15 Index found:  id_company
2023/09/01 14:02:15 Dropped index id_company
2023/09/01 14:02:15 Index found:  id_gender
2023/09/01 14:02:15 Dropped index id_gender
2023/09/01 14:02:24 Setting JSON docs in KV
2023/09/01 14:03:00 Created the secondary index id_company. Waiting for it become active
2023/09/01 14:03:00 Index is 4539957819743837279 now active
2023/09/01 14:03:01 Build command issued for the deferred indexes [1426452235199259294 2615433478115617537]
2023/09/01 14:03:02 Dropping the secondary index id_age
2023/09/01 14:03:02 Index dropped
2023/09/01 14:03:02 Dropping the secondary index id_gender
2023/09/01 14:03:18 Index dropped
2023/09/01 14:03:18 Build command issued for the deferred indexes [id_isActive], bucket: default, scope: _default, coll: _default
2023/09/01 14:03:18 Waiting for the index id_isActive to become active
2023/09/01 14:03:18 Waiting for index 6699762626151338066 to go active ...
2023/09/01 14:03:19 Waiting for index 6699762626151338066 to go active ...
2023/09/01 14:03:20 Waiting for index 6699762626151338066 to go active ...
2023/09/01 14:03:21 Waiting for index 6699762626151338066 to go active ...
2023/09/01 14:03:22 Waiting for index 6699762626151338066 to go active ...
2023/09/01 14:03:23 Waiting for index 6699762626151338066 to go active ...
2023/09/01 14:03:24 Waiting for index 6699762626151338066 to go active ...
2023/09/01 14:03:25 Waiting for index 6699762626151338066 to go active ...
2023/09/01 14:03:26 Waiting for index 6699762626151338066 to go active ...
2023/09/01 14:03:27 Waiting for index 6699762626151338066 to go active ...
2023/09/01 14:03:28 Waiting for index 6699762626151338066 to go active ...
2023/09/01 14:03:29 Waiting for index 6699762626151338066 to go active ...
2023/09/01 14:03:30 Waiting for index 6699762626151338066 to go active ...
2023/09/01 14:03:31 Waiting for index 6699762626151338066 to go active ...
2023/09/01 14:03:32 Waiting for index 6699762626151338066 to go active ...
2023/09/01 14:03:33 Waiting for index 6699762626151338066 to go active ...
2023/09/01 14:03:34 Waiting for index 6699762626151338066 to go active ...
2023/09/01 14:03:35 Waiting for index 6699762626151338066 to go active ...
2023/09/01 14:03:36 Index is 6699762626151338066 now active
2023/09/01 14:03:47 Using n1ql client
2023/09/01 14:03:48 Expected and Actual scan responses are the same
2023/09/01 14:03:48 Number of docScanResults and scanResults = 180000 and 180000
2023/09/01 14:03:48 Using n1ql client
2023/09/01 14:03:49 Expected and Actual scan responses are the same
2023/09/01 14:03:49 Number of docScanResults and scanResults = 180000 and 180000
--- PASS: TestDropMultipleBuildingDeferredIndexes (94.40s)
=== RUN   TestDropOneIndexSecondDeferBuilding
2023/09/01 14:03:49 In TestDropOneIndexSecondDeferBuilding()
2023/09/01 14:03:49 In DropAllSecondaryIndexes()
2023/09/01 14:03:49 Index found:  id_isActive
2023/09/01 14:03:49 Dropped index id_isActive
2023/09/01 14:03:49 Index found:  id_company
2023/09/01 14:03:49 Dropped index id_company
2023/09/01 14:03:52 Setting JSON docs in KV
2023/09/01 14:03:58 Build command issued for the deferred indexes [id_company], bucket: default, scope: _default, coll: _default
2023/09/01 14:03:58 Waiting for the index id_company to become active
2023/09/01 14:03:58 Waiting for index 10027820917522212970 to go active ...
2023/09/01 14:03:59 Waiting for index 10027820917522212970 to go active ...
2023/09/01 14:04:00 Waiting for index 10027820917522212970 to go active ...
2023/09/01 14:04:01 Waiting for index 10027820917522212970 to go active ...
2023/09/01 14:04:02 Waiting for index 10027820917522212970 to go active ...
2023/09/01 14:04:03 Waiting for index 10027820917522212970 to go active ...
2023/09/01 14:04:04 Waiting for index 10027820917522212970 to go active ...
2023/09/01 14:04:05 Waiting for index 10027820917522212970 to go active ...
2023/09/01 14:04:06 Waiting for index 10027820917522212970 to go active ...
2023/09/01 14:04:07 Waiting for index 10027820917522212970 to go active ...
2023/09/01 14:04:08 Waiting for index 10027820917522212970 to go active ...
2023/09/01 14:04:09 Waiting for index 10027820917522212970 to go active ...
2023/09/01 14:04:10 Waiting for index 10027820917522212970 to go active ...
2023/09/01 14:04:11 Waiting for index 10027820917522212970 to go active ...
2023/09/01 14:04:12 Waiting for index 10027820917522212970 to go active ...
2023/09/01 14:04:13 Waiting for index 10027820917522212970 to go active ...
2023/09/01 14:04:14 Waiting for index 10027820917522212970 to go active ...
2023/09/01 14:04:15 Waiting for index 10027820917522212970 to go active ...
2023/09/01 14:04:16 Index is 10027820917522212970 now active
2023/09/01 14:04:16 Build command issued for the deferred indexes [16784368154097440433]
2023/09/01 14:04:17 Dropping the secondary index id_company
2023/09/01 14:04:17 Index dropped
2023/09/01 14:04:22 Setting JSON docs in KV
2023/09/01 14:04:38 Waiting for index 16784368154097440433 to go active ...
2023/09/01 14:04:39 Waiting for index 16784368154097440433 to go active ...
2023/09/01 14:04:40 Waiting for index 16784368154097440433 to go active ...
2023/09/01 14:04:41 Waiting for index 16784368154097440433 to go active ...
2023/09/01 14:04:42 Index is 16784368154097440433 now active
2023/09/01 14:04:42 Build command issued for the deferred indexes [id_gender], bucket: default, scope: _default, coll: _default
2023/09/01 14:04:42 Waiting for the index id_gender to become active
2023/09/01 14:04:42 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:04:43 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:04:44 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:04:45 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:04:46 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:04:47 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:04:48 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:04:49 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:04:50 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:04:51 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:04:52 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:04:53 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:04:54 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:04:55 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:04:56 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:04:57 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:04:58 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:04:59 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:05:00 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:05:01 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:05:02 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:05:03 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:05:04 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:05:05 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:05:06 Waiting for index 13876648603284717118 to go active ...
2023/09/01 14:05:07 Index is 13876648603284717118 now active
2023/09/01 14:05:08 Using n1ql client
2023/09/01 14:05:08 Expected and Actual scan responses are the same
2023/09/01 14:05:08 Using n1ql client
2023/09/01 14:05:09 Expected and Actual scan responses are the same
--- PASS: TestDropOneIndexSecondDeferBuilding (79.99s)
=== RUN   TestDropSecondIndexSecondDeferBuilding
2023/09/01 14:05:09 In TestDropSecondIndexSecondDeferBuilding()
2023/09/01 14:05:09 In DropAllSecondaryIndexes()
2023/09/01 14:05:09 Index found:  id_gender
2023/09/01 14:05:09 Dropped index id_gender
2023/09/01 14:05:09 Index found:  id_age
2023/09/01 14:05:09 Dropped index id_age
2023/09/01 14:05:12 Setting JSON docs in KV
2023/09/01 14:05:18 Build command issued for the deferred indexes [id_company], bucket: default, scope: _default, coll: _default
2023/09/01 14:05:18 Waiting for the index id_company to become active
2023/09/01 14:05:18 Waiting for index 3065057682904275582 to go active ...
2023/09/01 14:05:19 Waiting for index 3065057682904275582 to go active ...
2023/09/01 14:05:20 Waiting for index 3065057682904275582 to go active ...
2023/09/01 14:05:21 Waiting for index 3065057682904275582 to go active ...
2023/09/01 14:05:22 Waiting for index 3065057682904275582 to go active ...
2023/09/01 14:05:23 Waiting for index 3065057682904275582 to go active ...
2023/09/01 14:05:24 Waiting for index 3065057682904275582 to go active ...
2023/09/01 14:05:25 Waiting for index 3065057682904275582 to go active ...
2023/09/01 14:05:26 Waiting for index 3065057682904275582 to go active ...
2023/09/01 14:05:27 Waiting for index 3065057682904275582 to go active ...
2023/09/01 14:05:28 Waiting for index 3065057682904275582 to go active ...
2023/09/01 14:05:29 Waiting for index 3065057682904275582 to go active ...
2023/09/01 14:05:30 Waiting for index 3065057682904275582 to go active ...
2023/09/01 14:05:31 Waiting for index 3065057682904275582 to go active ...
2023/09/01 14:05:32 Waiting for index 3065057682904275582 to go active ...
2023/09/01 14:05:33 Waiting for index 3065057682904275582 to go active ...
2023/09/01 14:05:34 Waiting for index 3065057682904275582 to go active ...
2023/09/01 14:05:35 Waiting for index 3065057682904275582 to go active ...
2023/09/01 14:05:36 Index is 3065057682904275582 now active
2023/09/01 14:05:36 Build command issued for the deferred indexes [11392745589656692743]
2023/09/01 14:05:37 Dropping the secondary index id_age
2023/09/01 14:05:37 Index dropped
2023/09/01 14:05:41 Setting JSON docs in KV
2023/09/01 14:05:49 Build command issued for the deferred indexes [id_gender], bucket: default, scope: _default, coll: _default
2023/09/01 14:05:49 Waiting for the index id_gender to become active
2023/09/01 14:05:49 Waiting for index 11233667796733001369 to go active ...
2023/09/01 14:05:50 Waiting for index 11233667796733001369 to go active ...
2023/09/01 14:05:51 Waiting for index 11233667796733001369 to go active ...
2023/09/01 14:05:52 Waiting for index 11233667796733001369 to go active ...
2023/09/01 14:05:53 Waiting for index 11233667796733001369 to go active ...
2023/09/01 14:05:54 Waiting for index 11233667796733001369 to go active ...
2023/09/01 14:05:55 Waiting for index 11233667796733001369 to go active ...
2023/09/01 14:05:56 Waiting for index 11233667796733001369 to go active ...
2023/09/01 14:05:57 Waiting for index 11233667796733001369 to go active ...
2023/09/01 14:05:58 Waiting for index 11233667796733001369 to go active ...
2023/09/01 14:05:59 Waiting for index 11233667796733001369 to go active ...
2023/09/01 14:06:00 Waiting for index 11233667796733001369 to go active ...
2023/09/01 14:06:01 Waiting for index 11233667796733001369 to go active ...
2023/09/01 14:06:02 Waiting for index 11233667796733001369 to go active ...
2023/09/01 14:06:03 Waiting for index 11233667796733001369 to go active ...
2023/09/01 14:06:04 Waiting for index 11233667796733001369 to go active ...
2023/09/01 14:06:05 Waiting for index 11233667796733001369 to go active ...
2023/09/01 14:06:06 Waiting for index 11233667796733001369 to go active ...
2023/09/01 14:06:07 Waiting for index 11233667796733001369 to go active ...
2023/09/01 14:06:08 Waiting for index 11233667796733001369 to go active ...
2023/09/01 14:06:09 Index is 11233667796733001369 now active
2023/09/01 14:06:09 Using n1ql client
2023/09/01 14:06:10 Expected and Actual scan responses are the same
2023/09/01 14:06:10 Using n1ql client
2023/09/01 14:06:11 Expected and Actual scan responses are the same
--- PASS: TestDropSecondIndexSecondDeferBuilding (62.29s)
=== RUN   TestCreateAfterDropWhileIndexBuilding
2023/09/01 14:06:11 In TestCreateAfterDropWhileIndexBuilding()
2023/09/01 14:06:11 In DropAllSecondaryIndexes()
2023/09/01 14:06:11 Index found:  id_gender
2023/09/01 14:06:12 Dropped index id_gender
2023/09/01 14:06:12 Index found:  id_company
2023/09/01 14:06:12 Dropped index id_company
2023/09/01 14:06:35 Setting JSON docs in KV
2023/09/01 14:07:33 Build command issued for the deferred indexes [16321615619341619914]
2023/09/01 14:07:34 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:35 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:36 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:37 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:38 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:39 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:40 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:41 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:42 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:43 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:44 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:45 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:46 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:47 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:48 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:49 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:50 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:51 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:52 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:53 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:54 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:55 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:56 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:57 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:58 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:07:59 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:08:00 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:08:01 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:08:02 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:08:03 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:08:04 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:08:05 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:08:06 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:08:07 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:08:08 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:08:09 Waiting for index 16321615619341619914 to go active ...
2023/09/01 14:08:10 Index is 16321615619341619914 now active
2023/09/01 14:08:10 Build command issued for the deferred indexes [14182834040272944819]
2023/09/01 14:08:11 Dropping the secondary index id_company
2023/09/01 14:08:12 Index dropped
2023/09/01 14:08:12 Dropping the secondary index id_age
2023/09/01 14:08:13 Index dropped
2023/09/01 14:08:19 Build command issued for the deferred indexes [id_gender], bucket: default, scope: _default, coll: _default
2023/09/01 14:08:19 Waiting for the index id_gender to become active
2023/09/01 14:08:19 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:20 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:21 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:22 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:23 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:24 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:25 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:26 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:27 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:28 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:29 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:30 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:31 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:32 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:33 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:34 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:35 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:36 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:37 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:38 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:39 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:40 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:41 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:42 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:43 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:44 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:45 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:46 Waiting for index 4948824754294218616 to go active ...
2023/09/01 14:08:47 Index is 4948824754294218616 now active
2023/09/01 14:08:48 Index is 4948824754294218616 now active
2023/09/01 14:08:49 Using n1ql client
2023/09/01 14:08:50 Expected and Actual scan responses are the same
--- PASS: TestCreateAfterDropWhileIndexBuilding (158.99s)
=== RUN   TestDropBuildingIndex1
2023/09/01 14:08:50 In TestDropBuildingIndex1()
2023/09/01 14:08:50 In DropAllSecondaryIndexes()
2023/09/01 14:08:50 Index found:  id_gender
2023/09/01 14:08:51 Dropped index id_gender
2023/09/01 14:08:54 Setting JSON docs in KV
2023/09/01 14:09:35 Created the secondary index id_company. Waiting for it become active
2023/09/01 14:09:35 Index is 12750260486285876141 now active
2023/09/01 14:10:07 Dropping the secondary index id_age
2023/09/01 14:10:07 Index dropped
2023/09/01 14:10:36 Created the secondary index id_age. Waiting for it become active
2023/09/01 14:10:36 Index is 7070851513188083048 now active
2023/09/01 14:10:38 Setting JSON docs in KV
2023/09/01 14:10:47 Using n1ql client
2023/09/01 14:10:48 Expected and Actual scan responses are the same
2023/09/01 14:10:48 Using n1ql client
2023/09/01 14:10:49 Expected and Actual scan responses are the same
--- PASS: TestDropBuildingIndex1 (118.43s)
=== RUN   TestDropBuildingIndex2
2023/09/01 14:10:49 In TestDropBuildingIndex2()
2023/09/01 14:10:49 In DropAllSecondaryIndexes()
2023/09/01 14:10:49 Index found:  id_company
2023/09/01 14:10:49 Dropped index id_company
2023/09/01 14:10:49 Index found:  id_age
2023/09/01 14:10:49 Dropped index id_age
2023/09/01 14:10:54 Setting JSON docs in KV
2023/09/01 14:11:35 Created the secondary index id_company. Waiting for it become active
2023/09/01 14:11:35 Index is 6719300510201368509 now active
2023/09/01 14:12:10 Dropping the secondary index id_company
2023/09/01 14:12:10 Index dropped
2023/09/01 14:12:10 Index is 1578663175851826636 now active
2023/09/01 14:12:41 Created the secondary index id_company. Waiting for it become active
2023/09/01 14:12:41 Index is 10197887120411071325 now active
2023/09/01 14:12:43 Setting JSON docs in KV
2023/09/01 14:12:55 Using n1ql client
2023/09/01 14:12:55 Expected and Actual scan responses are the same
2023/09/01 14:12:56 Using n1ql client
2023/09/01 14:12:57 Expected and Actual scan responses are the same
--- PASS: TestDropBuildingIndex2 (127.67s)
=== RUN   TestDropIndexWithDataLoad
2023/09/01 14:12:57 In TestDropIndexWithDataLoad()
2023/09/01 14:12:57 In DropAllSecondaryIndexes()
2023/09/01 14:12:57 Index found:  id_company
2023/09/01 14:12:57 Dropped index id_company
2023/09/01 14:12:57 Index found:  id_age
2023/09/01 14:12:57 Dropped index id_age
2023/09/01 14:12:59 Setting JSON docs in KV
2023/09/01 14:13:42 Created the secondary index id_company. Waiting for it become active
2023/09/01 14:13:42 Index is 7283419101313570061 now active
2023/09/01 14:14:17 Created the secondary index id_age. Waiting for it become active
2023/09/01 14:14:17 Index is 5007401541359238538 now active
2023/09/01 14:14:52 Created the secondary index id_gender. Waiting for it become active
2023/09/01 14:14:52 Index is 16515012280465683105 now active
2023/09/01 14:15:26 Created the secondary index id_isActive. Waiting for it become active
2023/09/01 14:15:26 Index is 1469937978289178460 now active
2023/09/01 14:15:33 Setting JSON docs in KV
2023/09/01 14:15:33 In LoadKVBucket
2023/09/01 14:15:33 Bucket name = default
2023/09/01 14:15:33 In DropIndexWhileKVLoad
2023/09/01 14:15:34 Dropping the secondary index id_company
2023/09/01 14:15:34 Index dropped
2023/09/01 14:16:02 Using n1ql client
2023/09/01 14:16:03 Expected and Actual scan responses are the same
2023/09/01 14:16:03 Number of docScanResults and scanResults = 96712 and 96712
2023/09/01 14:16:04 Using n1ql client
2023/09/01 14:16:07 Expected and Actual scan responses are the same
2023/09/01 14:16:07 Number of docScanResults and scanResults = 420000 and 420000
--- PASS: TestDropIndexWithDataLoad (190.79s)
=== RUN   TestDropAllIndexesWithDataLoad
2023/09/01 14:16:07 In TestDropAllIndexesWithDataLoad()
2023/09/01 14:16:07 In DropAllSecondaryIndexes()
2023/09/01 14:16:07 Index found:  id_gender
2023/09/01 14:16:07 Dropped index id_gender
2023/09/01 14:16:07 Index found:  id_age
2023/09/01 14:16:08 Dropped index id_age
2023/09/01 14:16:08 Index found:  id_isActive
2023/09/01 14:16:08 Dropped index id_isActive
2023/09/01 14:16:10 Setting JSON docs in KV
2023/09/01 14:16:52 Created the secondary index id_company. Waiting for it become active
2023/09/01 14:16:52 Index is 15283360776216507346 now active
2023/09/01 14:17:31 Created the secondary index id_age. Waiting for it become active
2023/09/01 14:17:31 Index is 1966322256287393016 now active
2023/09/01 14:18:12 Created the secondary index id_gender. Waiting for it become active
2023/09/01 14:18:12 Index is 5236510382601004486 now active
2023/09/01 14:18:48 Created the secondary index id_isActive. Waiting for it become active
2023/09/01 14:18:48 Index is 526927689954373477 now active
2023/09/01 14:18:55 Setting JSON docs in KV
2023/09/01 14:18:55 In LoadKVBucket
2023/09/01 14:18:55 Bucket name = default
2023/09/01 14:18:55 In DropIndexWhileKVLoad
2023/09/01 14:18:55 In DropIndexWhileKVLoad
2023/09/01 14:18:55 In DropIndexWhileKVLoad
2023/09/01 14:18:55 In DropIndexWhileKVLoad
2023/09/01 14:18:56 Dropping the secondary index id_gender
2023/09/01 14:18:56 Dropping the secondary index id_isActive
2023/09/01 14:18:56 Dropping the secondary index id_age
2023/09/01 14:18:56 Dropping the secondary index id_company
2023/09/01 14:18:56 Index dropped
2023/09/01 14:18:56 Index dropped
2023/09/01 14:18:56 Index dropped
2023/09/01 14:18:56 Index dropped
2023/09/01 14:19:15 Using n1ql client
2023/09/01 14:19:15 Scan failed as expected with error: Index Not Found - cause: GSI index id_company not found.
--- PASS: TestDropAllIndexesWithDataLoad (187.39s)
=== RUN   TestCreateBucket_AnotherIndexBuilding
2023/09/01 14:19:15 In TestCreateBucket_AnotherIndexBuilding()
2023/09/01 14:19:15 In DropAllSecondaryIndexes()
2023/09/01 14:19:53 Flushed the bucket default, Response body: 
2023/09/01 14:19:56 Modified parameters of bucket default, responseBody: 
2023/09/01 14:19:56 http://127.0.0.1:9000/pools/default/buckets/multi_buck2
2023/09/01 14:19:56 &{DELETE http://127.0.0.1:9000/pools/default/buckets/multi_buck2 HTTP/1.1 1 1 map[Authorization:[Basic QWRtaW5pc3RyYXRvcjphc2Rhc2Q=] Content-Type:[application/x-www-form-urlencoded; charset=UTF-8]]   0 [] false 127.0.0.1:9000 map[] map[]  map[]      0xc00012e000}
2023/09/01 14:19:56 &{404 Object Not Found 404 HTTP/1.1 1 1 map[Cache-Control:[no-cache,no-store,must-revalidate] Content-Length:[31] Content-Type:[text/plain] Date:[Fri, 01 Sep 2023 08:49:56 GMT] Expires:[Thu, 01 Jan 1970 00:00:00 GMT] Pragma:[no-cache] Server:[Couchbase Server] X-Content-Type-Options:[nosniff] X-Frame-Options:[DENY] X-Permitted-Cross-Domain-Policies:[none] X-Xss-Protection:[1; mode=block]] 0xc001af4700 31 [] false false map[] 0xc00645f500 }
2023/09/01 14:19:56 DeleteBucket failed for bucket multi_buck2 
2023/09/01 14:19:56 Deleted bucket multi_buck2, responseBody: Requested resource not found.
2023/09/01 14:20:11 Setting JSON docs in KV
2023/09/01 14:22:17 Created bucket multi_buck2, responseBody: 
2023/09/01 14:22:32 set: MCResponse status=TMPFAIL, opcode=SET, opaque=0, msg: 
--- FAIL: TestCreateBucket_AnotherIndexBuilding (197.06s)
panic: set: MCResponse status=TMPFAIL, opcode=SET, opaque=0, msg: 
 [recovered]
	panic: set: MCResponse status=TMPFAIL, opcode=SET, opaque=0, msg: 


goroutine 11104 [running]:
panic({0x1095680, 0xc03835b460})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/panic.go:987 +0x3bb fp=0xc007251940 sp=0xc007251880 pc=0x43cabb
testing.tRunner.func1.2({0x1095680, 0xc03835b460})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/testing/testing.go:1526 +0x24e fp=0xc0072519f0 sp=0xc007251940 pc=0x531b4e
testing.tRunner.func1()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/testing/testing.go:1529 +0x39f fp=0xc007251b58 sp=0xc0072519f0 pc=0x5315df
runtime.deferCallSave(0xc007251c28, 0xc007251f90?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/panic.go:796 +0x88 fp=0xc007251b68 sp=0xc007251b58 pc=0x43c6a8
runtime.runOpenDeferFrame(0xc00aae06e0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/panic.go:769 +0x1b4 fp=0xc007251bb0 sp=0xc007251b68 pc=0x43c4d4
panic({0x1095680, 0xc03835b460})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/panic.go:884 +0x213 fp=0xc007251c70 sp=0xc007251bb0 pc=0x43c913
log.Panicf({0x1260c7b?, 0x3?}, {0xc007251d20?, 0x7?, 0x10df0c0?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/log/log.go:391 +0x67 fp=0xc007251cb8 sp=0xc007251c70 pc=0x5e7967
github.com/couchbase/indexing/secondary/tests/framework/common.HandleError(...)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/framework/common/util.go:77
github.com/couchbase/indexing/secondary/tests/framework/kvutility.SetKeyValues(0x37e11d600?, {0x1266c93, 0xb}, {0x0, 0x0}, {0xc000406ad0, 0xe})
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/framework/kvutility/kvdataoperations.go:40 +0x34f fp=0xc007251e18 sp=0xc007251cb8 pc=0xd3078f
github.com/couchbase/indexing/secondary/tests/functionaltests.TestCreateBucket_AnotherIndexBuilding(0xc0a4c401a0)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/functionaltests/set07_system_test.go:1048 +0x4e5 fp=0xc007251f70 sp=0xc007251e18 pc=0xf89945
testing.tRunner(0xc0a4c401a0, 0x12e9958)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/testing/testing.go:1576 +0x10b fp=0xc007251fc0 sp=0xc007251f70 pc=0x53118b
testing.(*T).Run.func1()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/testing/testing.go:1629 +0x2a fp=0xc007251fe0 sp=0xc007251fc0 pc=0x5321ca
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc007251fe8 sp=0xc007251fe0 pc=0x473161
created by testing.(*T).Run
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/testing/testing.go:1629 +0x3ea

goroutine 1 [chan receive, 3 minutes]:
runtime.gopark(0xc014781000?, 0xc0aff92010?, 0xa0?, 0x21?, 0xc0054d7818?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc0054d7798 sp=0xc0054d7778 pc=0x43fd16
runtime.chanrecv(0xc08ca74070, 0xc0054d7897, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/chan.go:583 +0x49d fp=0xc0054d7828 sp=0xc0054d7798 pc=0x40ca7d
runtime.chanrecv1(0x1e138e0?, 0x1095680?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/chan.go:442 +0x18 fp=0xc0054d7850 sp=0xc0054d7828 pc=0x40c578
testing.(*T).Run(0xc0005f0000, {0x128e52a?, 0x530a05?}, 0x12e9958)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/testing/testing.go:1630 +0x405 fp=0xc0054d7910 sp=0xc0054d7850 pc=0x532045
testing.runTests.func1(0x1e15720?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/testing/testing.go:2036 +0x45 fp=0xc0054d7960 sp=0xc0054d7910 pc=0x534205
testing.tRunner(0xc0005f0000, 0xc009be9a78)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/testing/testing.go:1576 +0x10b fp=0xc0054d79b0 sp=0xc0054d7960 pc=0x53118b
testing.runTests(0xc0004c8c80?, {0x1d8fec0, 0xdd, 0xdd}, {0xc009be9b08?, 0xc009be9b10?, 0x1e14700?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/testing/testing.go:2034 +0x489 fp=0xc0054d7aa8 sp=0xc0054d79b0 pc=0x5340e9
testing.(*M).Run(0xc0004c8c80)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/testing/testing.go:1906 +0x63a fp=0xc0054d7cf0 sp=0xc0054d7aa8 pc=0x532a5a
github.com/couchbase/indexing/secondary/tests/functionaltests.TestMain(0xffffffffffffffff?)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/functionaltests/common_test.go:168 +0x1276 fp=0xc0054d7ec8 sp=0xc0054d7cf0 pc=0xf26d16
main.main()
	_testmain.go:489 +0x1d3 fp=0xc0054d7f80 sp=0xc0054d7ec8 pc=0x1006bf3
runtime.main()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:250 +0x207 fp=0xc0054d7fe0 sp=0xc0054d7f80 pc=0x43f8e7
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0054d7fe8 sp=0xc0054d7fe0 pc=0x473161

goroutine 2 [force gc (idle), 6 minutes]:
runtime.gopark(0xfaaa86df1b2?, 0x0?, 0x0?, 0x0?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00006cfb0 sp=0xc00006cf90 pc=0x43fd16
runtime.goparkunlock(...)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:387
runtime.forcegchelper()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:305 +0xb0 fp=0xc00006cfe0 sp=0xc00006cfb0 pc=0x43fb50
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00006cfe8 sp=0xc00006cfe0 pc=0x473161
created by runtime.init.6
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:293 +0x25

goroutine 3 [GC sweep wait]:
runtime.gopark(0x1e13d01?, 0x0?, 0x0?, 0x0?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00006d780 sp=0xc00006d760 pc=0x43fd16
runtime.goparkunlock(...)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:387
runtime.bgsweep(0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgcsweep.go:319 +0xde fp=0xc00006d7c8 sp=0xc00006d780 pc=0x42a0be
runtime.gcenable.func1()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:178 +0x26 fp=0xc00006d7e0 sp=0xc00006d7c8 pc=0x41f326
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00006d7e8 sp=0xc00006d7e0 pc=0x473161
created by runtime.gcenable
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:178 +0x6b

goroutine 4 [GC scavenge wait]:
runtime.gopark(0xfe13e8dca7f?, 0xa7fa3?, 0x0?, 0x0?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00006df70 sp=0xc00006df50 pc=0x43fd16
runtime.goparkunlock(...)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:387
runtime.(*scavengerState).park(0x1e14800)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgcscavenge.go:400 +0x53 fp=0xc00006dfa0 sp=0xc00006df70 pc=0x427f93
runtime.bgscavenge(0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgcscavenge.go:633 +0x65 fp=0xc00006dfc8 sp=0xc00006dfa0 pc=0x428585
runtime.gcenable.func2()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:179 +0x26 fp=0xc00006dfe0 sp=0xc00006dfc8 pc=0x41f2c6
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00006dfe8 sp=0xc00006dfe0 pc=0x473161
created by runtime.gcenable
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:179 +0xaa

goroutine 18 [finalizer wait]:
runtime.gopark(0x0?, 0x12ef580?, 0x20?, 0x20?, 0x2000000020?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00006c628 sp=0xc00006c608 pc=0x43fd16
runtime.runfinq()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mfinal.go:193 +0x107 fp=0xc00006c7e0 sp=0xc00006c628 pc=0x41e367
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00006c7e8 sp=0xc00006c7e0 pc=0x473161
created by runtime.createfing
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mfinal.go:163 +0x45

goroutine 19 [select, 61 minutes]:
runtime.gopark(0xc000068798?, 0x2?, 0x0?, 0x0?, 0xc000068784?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000068610 sp=0xc0000685f0 pc=0x43fd16
runtime.selectgo(0xc000068798, 0xc000068780, 0x0?, 0x0, 0x0?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc000068750 sp=0xc000068610 pc=0x4503be
github.com/couchbase/cbauth/cbauthimpl.(*tlsNotifier).loop(0xc00012c5e8)
	/opt/build/goproj/src/github.com/couchbase/cbauth/cbauthimpl/impl.go:426 +0x67 fp=0xc0000687c8 sp=0xc000068750 pc=0x7bf287
github.com/couchbase/cbauth/cbauthimpl.NewSVCForTest.func2()
	/opt/build/goproj/src/github.com/couchbase/cbauth/cbauthimpl/impl.go:700 +0x26 fp=0xc0000687e0 sp=0xc0000687c8 pc=0x7c0ce6
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000687e8 sp=0xc0000687e0 pc=0x473161
created by github.com/couchbase/cbauth/cbauthimpl.NewSVCForTest
	/opt/build/goproj/src/github.com/couchbase/cbauth/cbauthimpl/impl.go:700 +0x38c

goroutine 20 [select, 61 minutes]:
runtime.gopark(0xc000068f98?, 0x2?, 0x0?, 0x0?, 0xc000068f8c?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000068e08 sp=0xc000068de8 pc=0x43fd16
runtime.selectgo(0xc000068f98, 0xc000068f88, 0x0?, 0x0, 0x0?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc000068f48 sp=0xc000068e08 pc=0x4503be
github.com/couchbase/cbauth/cbauthimpl.(*cfgChangeNotifier).loop(0xc00012c600)
	/opt/build/goproj/src/github.com/couchbase/cbauth/cbauthimpl/impl.go:345 +0x85 fp=0xc000068fc8 sp=0xc000068f48 pc=0x7bece5
github.com/couchbase/cbauth/cbauthimpl.NewSVCForTest.func3()
	/opt/build/goproj/src/github.com/couchbase/cbauth/cbauthimpl/impl.go:701 +0x26 fp=0xc000068fe0 sp=0xc000068fc8 pc=0x7c0c86
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000068fe8 sp=0xc000068fe0 pc=0x473161
created by github.com/couchbase/cbauth/cbauthimpl.NewSVCForTest
	/opt/build/goproj/src/github.com/couchbase/cbauth/cbauthimpl/impl.go:701 +0x3da

goroutine 21 [IO wait]:
runtime.gopark(0x8?, 0xb?, 0x0?, 0x0?, 0x6?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc0000256e0 sp=0xc0000256c0 pc=0x43fd16
runtime.netpollblock(0x487505?, 0x40934f?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/netpoll.go:527 +0xf7 fp=0xc000025718 sp=0xc0000256e0 pc=0x438277
internal/poll.runtime_pollWait(0x7fc9c426af08, 0x72)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/netpoll.go:306 +0x89 fp=0xc000025738 sp=0xc000025718 pc=0x46d209
internal/poll.(*pollDesc).wait(0xc000040080?, 0xc0000ad000?, 0x0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc000025760 sp=0xc000025738 pc=0x4aa8f2
internal/poll.(*pollDesc).waitRead(...)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000040080, {0xc0000ad000, 0x1000, 0x1000})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_unix.go:167 +0x299 fp=0xc0000257f8 sp=0xc000025760 pc=0x4abcd9
net.(*netFD).Read(0xc000040080, {0xc0000ad000?, 0xc0000258c0?, 0x57b725?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/fd_posix.go:55 +0x29 fp=0xc000025840 sp=0xc0000257f8 pc=0x688e49
net.(*conn).Read(0xc000014010, {0xc0000ad000?, 0xc02cde63b0?, 0x1?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/net.go:183 +0x45 fp=0xc000025888 sp=0xc000025840 pc=0x699085
net.(*TCPConn).Read(0xc00575ab90?, {0xc0000ad000?, 0x57c556?, 0x109abc0?})
	:1 +0x29 fp=0xc0000258b8 sp=0xc000025888 pc=0x6ab8e9
bufio.(*Reader).Read(0xc000098120, {0xc0000cc001, 0x5ff, 0xc000025900?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/bufio/bufio.go:237 +0x1bb fp=0xc0000258f0 sp=0xc0000258b8 pc=0x538fdb
github.com/couchbase/cbauth/revrpc.(*minirwc).Read(0x1?, {0xc0000cc001?, 0x2?, 0x1?})
	/opt/build/goproj/src/github.com/couchbase/cbauth/revrpc/revrpc.go:124 +0x25 fp=0xc000025920 sp=0xc0000258f0 pc=0x8272c5
encoding/json.(*Decoder).refill(0xc0000c8000)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/encoding/json/stream.go:165 +0x188 fp=0xc000025970 sp=0xc000025920 pc=0x58e528
encoding/json.(*Decoder).readValue(0xc0000c8000)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/encoding/json/stream.go:140 +0xbb fp=0xc0000259c0 sp=0xc000025970 pc=0x58e11b
encoding/json.(*Decoder).Decode(0xc0000c8000, {0x10ccee0, 0xc0000984a0})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/encoding/json/stream.go:63 +0x78 fp=0xc0000259f0 sp=0xc0000259c0 pc=0x58dd58
net/rpc/jsonrpc.(*serverCodec).ReadRequestHeader(0xc000098480, 0xc0000580e0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/rpc/jsonrpc/server.go:66 +0x85 fp=0xc000025a30 sp=0xc0000259f0 pc=0x826485
github.com/couchbase/cbauth/revrpc.(*jsonServerCodec).ReadRequestHeader(0xc00007e080?, 0x41378a?)
	:1 +0x2a fp=0xc000025a50 sp=0xc000025a30 pc=0x82960a
net/rpc.(*Server).readRequestHeader(0xc00007e080, {0x1426750, 0xc00007a410})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/rpc/server.go:588 +0x66 fp=0xc000025b20 sp=0xc000025a50 pc=0x824686
net/rpc.(*Server).readRequest(0x0?, {0x1426750, 0xc00007a410})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/rpc/server.go:548 +0x3b fp=0xc000025bf8 sp=0xc000025b20 pc=0x8241db
net/rpc.(*Server).ServeCodec(0xc00007e080, {0x1426750?, 0xc00007a410})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/rpc/server.go:463 +0x87 fp=0xc000025cf0 sp=0xc000025bf8 pc=0x823727
github.com/couchbase/cbauth/revrpc.(*Service).Run(0x0?, 0xc000157f80)
	/opt/build/goproj/src/github.com/couchbase/cbauth/revrpc/revrpc.go:286 +0xa54 fp=0xc000025f28 sp=0xc000025cf0 pc=0x8283b4
github.com/couchbase/cbauth/revrpc.BabysitService(0x0?, 0x0?, {0x141b020?, 0xc00012c618?})
	/opt/build/goproj/src/github.com/couchbase/cbauth/revrpc/revrpc.go:401 +0x58 fp=0xc000025f60 sp=0xc000025f28 pc=0x828c98
github.com/couchbase/cbauth.runRPCForSvc(0x0?, 0x0?, 0x0?)
	/opt/build/goproj/src/github.com/couchbase/cbauth/default.go:119 +0x45 fp=0xc000025fa0 sp=0xc000025f60 pc=0x835645
github.com/couchbase/cbauth.startDefault.func1()
	/opt/build/goproj/src/github.com/couchbase/cbauth/default.go:132 +0x29 fp=0xc000025fe0 sp=0xc000025fa0 pc=0x835889
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000025fe8 sp=0xc000025fe0 pc=0x473161
created by github.com/couchbase/cbauth.startDefault
	/opt/build/goproj/src/github.com/couchbase/cbauth/default.go:131 +0x13c

goroutine 6 [GC worker (idle)]:
runtime.gopark(0xfe12ab80fae?, 0x3?, 0x7e?, 0x60?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000346750 sp=0xc000346730 pc=0x43fd16
runtime.gcBgMarkWorker()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc0003467e0 sp=0xc000346750 pc=0x421091
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0003467e8 sp=0xc0003467e0 pc=0x473161
created by runtime.gcBgMarkStartWorkers
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1199 +0x25

goroutine 22 [chan receive, 1 minutes]:
runtime.gopark(0xc000400000?, 0x0?, 0x0?, 0x0?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000069ef8 sp=0xc000069ed8 pc=0x43fd16
runtime.chanrecv(0xc000122840, 0x0, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/chan.go:583 +0x49d fp=0xc000069f88 sp=0xc000069ef8 pc=0x40ca7d
runtime.chanrecv1(0x0?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/chan.go:442 +0x18 fp=0xc000069fb0 sp=0xc000069f88 pc=0x40c578
github.com/couchbase/regulator/utils/logging.duplicateFlushLoop()
	/opt/build/goproj/src/github.com/couchbase/regulator/utils/logging/logger_golog.go:253 +0x25 fp=0xc000069fd0 sp=0xc000069fb0 pc=0xb3e525
github.com/couchbase/regulator/utils/logging.init.0.func1()
	/opt/build/goproj/src/github.com/couchbase/regulator/utils/logging/logger_golog.go:49 +0x25 fp=0xc000069fe0 sp=0xc000069fd0 pc=0xb40f65
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000069fe8 sp=0xc000069fe0 pc=0x473161
created by github.com/couchbase/regulator/utils/logging.init.0
	/opt/build/goproj/src/github.com/couchbase/regulator/utils/logging/logger_golog.go:49 +0x7e

goroutine 37 [GC worker (idle)]:
runtime.gopark(0xfe133ea729d?, 0x3?, 0x0?, 0xc5?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000346f50 sp=0xc000346f30 pc=0x43fd16
runtime.gcBgMarkWorker()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc000346fe0 sp=0xc000346f50 pc=0x421091
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000346fe8 sp=0xc000346fe0 pc=0x473161
created by runtime.gcBgMarkStartWorkers
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1199 +0x25

goroutine 35 [select, 61 minutes]:
runtime.gopark(0xc000085f68?, 0x4?, 0x4a?, 0xd?, 0xc000085db0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000085c00 sp=0xc000085be0 pc=0x43fd16
runtime.selectgo(0xc000085f68, 0xc000085da8, 0xc0001fe2c0?, 0x0, 0x1?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc000085d40 sp=0xc000085c00 pc=0x4503be
net/http.(*persistConn).readLoop(0xc000328000)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2227 +0xd85 fp=0xc000085fc8 sp=0xc000085d40 pc=0x7a34c5
net/http.(*Transport).dialConn.func5()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x26 fp=0xc000085fe0 sp=0xc000085fc8 pc=0x7a1086
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000085fe8 sp=0xc000085fe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x16ea

goroutine 36 [select, 61 minutes]:
runtime.gopark(0xc000316f90?, 0x2?, 0xf8?, 0x6d?, 0xc000316f34?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000316db0 sp=0xc000316d90 pc=0x43fd16
runtime.selectgo(0xc000316f90, 0xc000316f30, 0xc00033e040?, 0x0, 0xc0000a4ea0?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc000316ef0 sp=0xc000316db0 pc=0x4503be
net/http.(*persistConn).writeLoop(0xc000328000)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2410 +0xf2 fp=0xc000316fc8 sp=0xc000316ef0 pc=0x7a45b2
net/http.(*Transport).dialConn.func6()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x26 fp=0xc000316fe0 sp=0xc000316fc8 pc=0x7a1026
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000316fe8 sp=0xc000316fe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x173d

goroutine 50 [select, 61 minutes]:
runtime.gopark(0xc000157cc8?, 0x3?, 0x20?, 0x0?, 0xc000157c42?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000157aa0 sp=0xc000157a80 pc=0x43fd16
runtime.selectgo(0xc000157cc8, 0xc000157c3c, 0xc00012f160?, 0x0, 0x3?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc000157be0 sp=0xc000157aa0 pc=0x4503be
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0x0?, {0x1275f2f, 0x17}, 0xc000157f90, 0xc00040a060)
	/opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:310 +0x429 fp=0xc000157f68 sp=0xc000157be0 pc=0xb32c09
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(...)
	/opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:268
github.com/couchbase/cbauth/metakv.RunObserveChildren({0x1275f2f?, 0xc0003467d0?}, 0x7a1026?, 0xc000328000?)
	/opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:398 +0x58 fp=0xc000157fb0 sp=0xc000157f68 pc=0xb33158
github.com/couchbase/query/sequences.init.0.func1()
	/opt/build/goproj/src/github.com/couchbase/query/sequences/sequences.go:74 +0x36 fp=0xc000157fe0 sp=0xc000157fb0 pc=0xbdda36
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000157fe8 sp=0xc000157fe0 pc=0x473161
created by github.com/couchbase/query/sequences.init.0
	/opt/build/goproj/src/github.com/couchbase/query/sequences/sequences.go:74 +0x1aa

goroutine 24 [IO wait, 61 minutes]:
runtime.gopark(0xc00015ba58?, 0xb?, 0x0?, 0x0?, 0x7?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00015b9a0 sp=0xc00015b980 pc=0x43fd16
runtime.netpollblock(0x487505?, 0x40934f?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/netpoll.go:527 +0xf7 fp=0xc00015b9d8 sp=0xc00015b9a0 pc=0x438277
internal/poll.runtime_pollWait(0x7fc9c426ae18, 0x72)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/netpoll.go:306 +0x89 fp=0xc00015b9f8 sp=0xc00015b9d8 pc=0x46d209
internal/poll.(*pollDesc).wait(0xc00033a000?, 0xc000340000?, 0x0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc00015ba20 sp=0xc00015b9f8 pc=0x4aa8f2
internal/poll.(*pollDesc).waitRead(...)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc00033a000, {0xc000340000, 0x1000, 0x1000})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_unix.go:167 +0x299 fp=0xc00015bab8 sp=0xc00015ba20 pc=0x4abcd9
net.(*netFD).Read(0xc00033a000, {0xc000340000?, 0x1095680?, 0xc00015bb40?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/fd_posix.go:55 +0x29 fp=0xc00015bb00 sp=0xc00015bab8 pc=0x688e49
net.(*conn).Read(0xc00032c008, {0xc000340000?, 0xc0003c00a2?, 0x5?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/net.go:183 +0x45 fp=0xc00015bb48 sp=0xc00015bb00 pc=0x699085
net.(*TCPConn).Read(0xc000134e50?, {0xc000340000?, 0x0?, 0xa3?})
	:1 +0x29 fp=0xc00015bb78 sp=0xc00015bb48 pc=0x6ab8e9
net/http.(*persistConn).Read(0xc000328000, {0xc000340000?, 0x0?, 0x0?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1943 +0x4e fp=0xc00015bbd8 sp=0xc00015bb78 pc=0x7a1ace
bufio.(*Reader).fill(0xc00031e180)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/bufio/bufio.go:106 +0xff fp=0xc00015bc10 sp=0xc00015bbd8 pc=0x5389ff
bufio.(*Reader).ReadSlice(0xc00031e180, 0xb?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/bufio/bufio.go:372 +0x2f fp=0xc00015bc60 sp=0xc00015bc10 pc=0x5395ef
net/http/internal.readChunkLine(0x97?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/internal/chunked.go:129 +0x25 fp=0xc00015bcb0 sp=0xc00015bc60 pc=0x73ea85
net/http/internal.(*chunkedReader).beginChunk(0xc00018bce0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/internal/chunked.go:48 +0x28 fp=0xc00015bce0 sp=0xc00015bcb0 pc=0x73e4e8
net/http/internal.(*chunkedReader).Read(0xc00018bce0, {0xc0003c0000?, 0xc0001555f0?, 0xc00018bda0?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/internal/chunked.go:98 +0x156 fp=0xc00015bd60 sp=0xc00015bce0 pc=0x73e7b6
net/http.(*body).readLocked(0xc0001fe2c0, {0xc0003c0000?, 0xc0003be090?, 0x11793c0?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transfer.go:839 +0x3c fp=0xc00015bdb0 sp=0xc00015bd60 pc=0x79695c
net/http.(*body).Read(0x0?, {0xc0003c0000?, 0x57b3de?, 0x0?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transfer.go:831 +0x125 fp=0xc00015be28 sp=0xc00015bdb0 pc=0x796825
net/http.(*bodyEOFSignal).Read(0xc0001fe300, {0xc0003c0000, 0x200, 0x200})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2792 +0x142 fp=0xc00015bea8 sp=0xc00015be28 pc=0x7a61c2
encoding/json.(*Decoder).refill(0xc0003be000)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/encoding/json/stream.go:165 +0x188 fp=0xc00015bef8 sp=0xc00015bea8 pc=0x58e528
encoding/json.(*Decoder).readValue(0xc0003be000)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/encoding/json/stream.go:140 +0xbb fp=0xc00015bf48 sp=0xc00015bef8 pc=0x58e11b
encoding/json.(*Decoder).Decode(0xc0003be000, {0x1065120, 0xc000134e10})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/encoding/json/stream.go:63 +0x78 fp=0xc00015bf78 sp=0xc00015bf48 pc=0x58dd58
github.com/couchbase/cbauth/metakv.doRunObserveChildren.func1()
	/opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:293 +0xc5 fp=0xc00015bfe0 sp=0xc00015bf78 pc=0xb33045
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00015bfe8 sp=0xc00015bfe0 pc=0x473161
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:289 +0x2eb

goroutine 130 [select, 59 minutes]:
runtime.gopark(0xc000333f68?, 0x4?, 0x3?, 0x0?, 0xc000333db0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000333c00 sp=0xc000333be0 pc=0x43fd16
runtime.selectgo(0xc000333f68, 0xc000333da8, 0xc005398bc0?, 0x0, 0xc000349501?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc000333d40 sp=0xc000333c00 pc=0x4503be
net/http.(*persistConn).readLoop(0xc0000b6c60)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2227 +0xd85 fp=0xc000333fc8 sp=0xc000333d40 pc=0x7a34c5
net/http.(*Transport).dialConn.func5()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x26 fp=0xc000333fe0 sp=0xc000333fc8 pc=0x7a1086
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000333fe8 sp=0xc000333fe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x16ea

goroutine 8 [chan receive, 61 minutes]:
runtime.gopark(0x7fc9c426ab48?, 0x0?, 0x0?, 0x0?, 0xc000342f18?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000342e78 sp=0xc000342e58 pc=0x43fd16
runtime.chanrecv(0xc00031e480, 0xc000342f90, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/chan.go:583 +0x49d fp=0xc000342f08 sp=0xc000342e78 pc=0x40ca7d
runtime.chanrecv2(0xc00012cc00?, 0x16?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/chan.go:447 +0x18 fp=0xc000342f30 sp=0xc000342f08 pc=0x40c5b8
github.com/couchbase/goutils/systemeventlog.(*SystemEventLoggerImpl).logEvents(0xc0001fdd50)
	/opt/build/goproj/src/github.com/couchbase/goutils/systemeventlog/system_event_logger.go:186 +0xb7 fp=0xc000342fc8 sp=0xc000342f30 pc=0xd396f7
github.com/couchbase/goutils/systemeventlog.NewSystemEventLogger.func1()
	/opt/build/goproj/src/github.com/couchbase/goutils/systemeventlog/system_event_logger.go:125 +0x26 fp=0xc000342fe0 sp=0xc000342fc8 pc=0xd39086
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000342fe8 sp=0xc000342fe0 pc=0x473161
created by github.com/couchbase/goutils/systemeventlog.NewSystemEventLogger
	/opt/build/goproj/src/github.com/couchbase/goutils/systemeventlog/system_event_logger.go:125 +0x1d6

goroutine 25 [GC worker (idle)]:
runtime.gopark(0x1e515a0?, 0x1?, 0xbb?, 0xb5?, 0xc00006afd0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00006af50 sp=0xc00006af30 pc=0x43fd16
runtime.gcBgMarkWorker()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc00006afe0 sp=0xc00006af50 pc=0x421091
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00006afe8 sp=0xc00006afe0 pc=0x473161
created by runtime.gcBgMarkStartWorkers
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1199 +0x25

goroutine 26 [GC worker (idle)]:
runtime.gopark(0xfe133b7db5d?, 0x3?, 0xfc?, 0xc1?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00006b750 sp=0xc00006b730 pc=0x43fd16
runtime.gcBgMarkWorker()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc00006b7e0 sp=0xc00006b750 pc=0x421091
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00006b7e8 sp=0xc00006b7e0 pc=0x473161
created by runtime.gcBgMarkStartWorkers
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1199 +0x25

goroutine 81 [select, 60 minutes]:
runtime.gopark(0xc00002ef90?, 0x2?, 0xf8?, 0xed?, 0xc00002ef34?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00002edb0 sp=0xc00002ed90 pc=0x43fd16
runtime.selectgo(0xc00002ef90, 0xc00002ef30, 0xc000448000?, 0x0, 0xc0000a40c0?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc00002eef0 sp=0xc00002edb0 pc=0x4503be
net/http.(*persistConn).writeLoop(0xc000328240)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2410 +0xf2 fp=0xc00002efc8 sp=0xc00002eef0 pc=0x7a45b2
net/http.(*Transport).dialConn.func6()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x26 fp=0xc00002efe0 sp=0xc00002efc8 pc=0x7a1026
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00002efe8 sp=0xc00002efe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x173d

goroutine 45 [select]:
runtime.gopark(0xc0000e3ba0?, 0x3?, 0x8a?, 0x37?, 0xc0000e3b1a?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc0000e3978 sp=0xc0000e3958 pc=0x43fd16
runtime.selectgo(0xc0000e3ba0, 0xc0000e3b14, 0x0?, 0x0, 0x0?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc0000e3ab8 sp=0xc0000e3978 pc=0x4503be
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc000196620?, {0x127bf71, 0x1b}, 0xc0000e3e68, 0xc000056540)
	/opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:310 +0x429 fp=0xc0000e3e40 sp=0xc0000e3ab8 pc=0xb32c09
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(...)
	/opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:268
github.com/couchbase/cbauth/metakv.RunObserveChildren({0x127bf71?, 0x0?}, 0x0?, 0x0?)
	/opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:398 +0x58 fp=0xc0000e3e88 sp=0xc0000e3e40 pc=0xb33158
github.com/couchbase/indexing/secondary/manager/common.(*CommandListener).ListenTokens.func2.1(0x0?, {0x0?, 0x0?})
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/manager/common/token.go:1626 +0xc7 fp=0xc0000e3f00 sp=0xc0000e3e88 pc=0xd3e8c7
github.com/couchbase/indexing/secondary/common.(*RetryHelper).Run(0xc0000e3fa0)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/common/retry_helper.go:36 +0x83 fp=0xc0000e3f38 sp=0xc0000e3f00 pc=0xc79063
github.com/couchbase/indexing/secondary/manager/common.(*CommandListener).ListenTokens.func2()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/manager/common/token.go:1631 +0xdf fp=0xc0000e3fe0 sp=0xc0000e3f38 pc=0xd3e77f
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0000e3fe8 sp=0xc0000e3fe0 pc=0x473161
created by github.com/couchbase/indexing/secondary/manager/common.(*CommandListener).ListenTokens
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/manager/common/token.go:1619 +0xaf

goroutine 44 [select]:
runtime.gopark(0xc000343798?, 0x2?, 0x91?, 0xe4?, 0xc00034378c?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000343618 sp=0xc0003435f8 pc=0x43fd16
runtime.selectgo(0xc000343798, 0xc000343788, 0x0?, 0x0, 0x0?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc000343758 sp=0xc000343618 pc=0x4503be
github.com/couchbase/indexing/secondary/queryport/client.(*schedTokenMonitor).updater(0xc0003fc690)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:2410 +0x92 fp=0xc0003437c8 sp=0xc000343758 pc=0xea9652
github.com/couchbase/indexing/secondary/queryport/client.newSchedTokenMonitor.func1()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:2203 +0x26 fp=0xc0003437e0 sp=0xc0003437c8 pc=0xea8026
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0003437e8 sp=0xc0003437e0 pc=0x473161
created by github.com/couchbase/indexing/secondary/queryport/client.newSchedTokenMonitor
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:2203 +0x296

goroutine 67 [select, 60 minutes]:
runtime.gopark(0xc000318f68?, 0x4?, 0x3?, 0x0?, 0xc000318db0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000318c00 sp=0xc000318be0 pc=0x43fd16
runtime.selectgo(0xc000318f68, 0xc000318da8, 0xc00033ee80?, 0x0, 0x1?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc000318d40 sp=0xc000318c00 pc=0x4503be
net/http.(*persistConn).readLoop(0xc0000b6240)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2227 +0xd85 fp=0xc000318fc8 sp=0xc000318d40 pc=0x7a34c5
net/http.(*Transport).dialConn.func5()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x26 fp=0xc000318fe0 sp=0xc000318fc8 pc=0x7a1086
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000318fe8 sp=0xc000318fe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x16ea

goroutine 68 [select, 60 minutes]:
runtime.gopark(0xc000319f90?, 0x2?, 0xf8?, 0x9d?, 0xc000319f34?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000319db0 sp=0xc000319d90 pc=0x43fd16
runtime.selectgo(0xc000319f90, 0xc000319f30, 0xc00007e500?, 0x0, 0xc000417230?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc000319ef0 sp=0xc000319db0 pc=0x4503be
net/http.(*persistConn).writeLoop(0xc0000b6240)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2410 +0xf2 fp=0xc000319fc8 sp=0xc000319ef0 pc=0x7a45b2
net/http.(*Transport).dialConn.func6()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x26 fp=0xc000319fe0 sp=0xc000319fc8 pc=0x7a1026
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000319fe8 sp=0xc000319fe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x173d

goroutine 95 [select]:
runtime.gopark(0xc000344790?, 0x2?, 0x50?, 0x9d?, 0xc000344774?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc0003445f8 sp=0xc0003445d8 pc=0x43fd16
runtime.selectgo(0xc000344790, 0xc000344770, 0x0?, 0x0, 0x2?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc000344738 sp=0xc0003445f8 pc=0x4503be
github.com/couchbase/indexing/secondary/manager/client.(*watcher).timeoutChecker(0xc00039c240)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:6686 +0xcd fp=0xc0003447c8 sp=0xc000344738 pc=0xe781ad
github.com/couchbase/indexing/secondary/manager/client.(*watcher).startTimer.func1()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:6677 +0x26 fp=0xc0003447e0 sp=0xc0003447c8 pc=0xe780a6
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0003447e8 sp=0xc0003447e0 pc=0x473161
created by github.com/couchbase/indexing/secondary/manager/client.(*watcher).startTimer
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:6677 +0x8d

goroutine 140 [select]:
runtime.gopark(0xc000343f98?, 0x2?, 0x91?, 0xe4?, 0xc000343f8c?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000343e18 sp=0xc000343df8 pc=0x43fd16
runtime.selectgo(0xc000343f98, 0xc000343f88, 0x1000000459100?, 0x0, 0xc000165580?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc000343f58 sp=0xc000343e18 pc=0x4503be
github.com/couchbase/indexing/secondary/queryport/client.(*schedTokenMonitor).updater(0xc001a618f0)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:2410 +0x92 fp=0xc000343fc8 sp=0xc000343f58 pc=0xea9652
github.com/couchbase/indexing/secondary/queryport/client.newSchedTokenMonitor.func1()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:2203 +0x26 fp=0xc000343fe0 sp=0xc000343fc8 pc=0xea8026
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000343fe8 sp=0xc000343fe0 pc=0x473161
created by github.com/couchbase/indexing/secondary/queryport/client.newSchedTokenMonitor
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:2203 +0x296

goroutine 1203 [IO wait, 51 minutes]:
runtime.gopark(0x3?, 0xb?, 0x0?, 0x0?, 0x22?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc004dd5c20 sp=0xc004dd5c00 pc=0x43fd16
runtime.netpollblock(0x487505?, 0x40934f?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/netpoll.go:527 +0xf7 fp=0xc004dd5c58 sp=0xc004dd5c20 pc=0x438277
internal/poll.runtime_pollWait(0x7fc9bc97d518, 0x72)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/netpoll.go:306 +0x89 fp=0xc004dd5c78 sp=0xc004dd5c58 pc=0x46d209
internal/poll.(*pollDesc).wait(0xc00a308f00?, 0xc0031be000?, 0x0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc004dd5ca0 sp=0xc004dd5c78 pc=0x4aa8f2
internal/poll.(*pollDesc).waitRead(...)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc00a308f00, {0xc0031be000, 0x1000, 0x1000})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_unix.go:167 +0x299 fp=0xc004dd5d38 sp=0xc004dd5ca0 pc=0x4abcd9
net.(*netFD).Read(0xc00a308f00, {0xc0031be000?, 0x0?, 0xc004dd5da8?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/fd_posix.go:55 +0x29 fp=0xc004dd5d80 sp=0xc004dd5d38 pc=0x688e49
net.(*conn).Read(0xc00032c860, {0xc0031be000?, 0xc00abbdfb0?, 0xc00b8e0900?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/net.go:183 +0x45 fp=0xc004dd5dc8 sp=0xc004dd5d80 pc=0x699085
net.(*TCPConn).Read(0xc0000fd400?, {0xc0031be000?, 0xd10dc2?, 0xc004b29a70?})
	:1 +0x29 fp=0xc004dd5df8 sp=0xc004dd5dc8 pc=0x6ab8e9
bufio.(*Reader).Read(0xc00af7d200, {0xc00af4b500, 0x18, 0xc00ac1bb00?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/bufio/bufio.go:237 +0x1bb fp=0xc004dd5e30 sp=0xc004dd5df8 pc=0x538fdb
gopkg.in/couchbase/gocbcore%2ev7.(*memdTcpConn).readFullBuffer(0xc00ad8c960, {0xc00af4b500?, 0xc00a5a9c00?, 0xc00ac1bb00?})
	/opt/build/go/pkg/mod/gopkg.in/couchbase/gocbcore.v7@v7.1.18/memdconn.go:143 +0x6d fp=0xc004dd5e78 sp=0xc004dd5e30 pc=0xd0d10d
gopkg.in/couchbase/gocbcore%2ev7.(*memdTcpConn).ReadPacket(0xc00ad8c960, 0xc00ac1bb00)
	/opt/build/go/pkg/mod/gopkg.in/couchbase/gocbcore.v7@v7.1.18/memdconn.go:159 +0x36 fp=0xc004dd5ed8 sp=0xc004dd5e78 pc=0xd0d1b6
gopkg.in/couchbase/gocbcore%2ev7.(*memdClient).run.func2()
	/opt/build/go/pkg/mod/gopkg.in/couchbase/gocbcore.v7@v7.1.18/memdclient.go:286 +0x102 fp=0xc004dd5fe0 sp=0xc004dd5ed8 pc=0xd0a882
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc004dd5fe8 sp=0xc004dd5fe0 pc=0x473161
created by gopkg.in/couchbase/gocbcore%2ev7.(*memdClient).run
	/opt/build/go/pkg/mod/gopkg.in/couchbase/gocbcore.v7@v7.1.18/memdclient.go:279 +0x16c

goroutine 3174 [select]:
runtime.gopark(0xc007275f90?, 0x2?, 0xc0?, 0x74?, 0xc007275f4c?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc007275dc8 sp=0xc007275da8 pc=0x43fd16
runtime.selectgo(0xc007275f90, 0xc007275f48, 0xc000bdc960?, 0x0, 0x100010100000000?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc007275f08 sp=0xc007275dc8 pc=0x4503be
github.com/couchbase/gometa/protocol.(*Follower).startListener(0xc00764b960)
	/opt/build/goproj/src/github.com/couchbase/gometa/protocol/follower.go:149 +0xe5 fp=0xc007275fc8 sp=0xc007275f08 pc=0xdee685
github.com/couchbase/gometa/protocol.(*Follower).Start.func1()
	/opt/build/goproj/src/github.com/couchbase/gometa/protocol/follower.go:76 +0x26 fp=0xc007275fe0 sp=0xc007275fc8 pc=0xdee286
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc007275fe8 sp=0xc007275fe0 pc=0x473161
created by github.com/couchbase/gometa/protocol.(*Follower).Start
	/opt/build/goproj/src/github.com/couchbase/gometa/protocol/follower.go:76 +0x56

goroutine 15233 [IO wait]:
runtime.gopark(0x1bb4?, 0xb?, 0x0?, 0x0?, 0x5a?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc007d578c8 sp=0xc007d578a8 pc=0x43fd16
runtime.netpollblock(0x487505?, 0x40934f?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/netpoll.go:527 +0xf7 fp=0xc007d57900 sp=0xc007d578c8 pc=0x438277
internal/poll.runtime_pollWait(0x7fc9bc97d428, 0x72)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/netpoll.go:306 +0x89 fp=0xc007d57920 sp=0xc007d57900 pc=0x46d209
internal/poll.(*pollDesc).wait(0xc0186fd280?, 0xc0b0523000?, 0x0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc007d57948 sp=0xc007d57920 pc=0x4aa8f2
internal/poll.(*pollDesc).waitRead(...)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0186fd280, {0xc0b0523000, 0x1000, 0x1000})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_unix.go:167 +0x299 fp=0xc007d579e0 sp=0xc007d57948 pc=0x4abcd9
net.(*netFD).Read(0xc0186fd280, {0xc0b0523000?, 0x58c8b6?, 0xc0001934f8?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/fd_posix.go:55 +0x29 fp=0xc007d57a28 sp=0xc007d579e0 pc=0x688e49
net.(*conn).Read(0xc0aff92448, {0xc0b0523000?, 0xc0001934f8?, 0xc0001934f8?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/net.go:183 +0x45 fp=0xc007d57a70 sp=0xc007d57a28 pc=0x699085
net.(*TCPConn).Read(0x57b3c3?, {0xc0b0523000?, 0xc007844000?, 0xc007d57b28?})
	:1 +0x29 fp=0xc007d57aa0 sp=0xc007d57a70 pc=0x6ab8e9
net/http.(*persistConn).Read(0xc0b0377560, {0xc0b0523000?, 0x1f7e?, 0xc0001934f8?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1943 +0x4e fp=0xc007d57b00 sp=0xc007d57aa0 pc=0x7a1ace
bufio.(*Reader).fill(0xc026c14cc0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/bufio/bufio.go:106 +0xff fp=0xc007d57b38 sp=0xc007d57b00 pc=0x5389ff
bufio.(*Reader).ReadSlice(0xc026c14cc0, 0x1?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/bufio/bufio.go:372 +0x2f fp=0xc007d57b88 sp=0xc007d57b38 pc=0x5395ef
net/http/internal.readChunkLine(0x9c?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/internal/chunked.go:129 +0x25 fp=0xc007d57bd8 sp=0xc007d57b88 pc=0x73ea85
net/http/internal.(*chunkedReader).beginChunk(0xc001758540)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/internal/chunked.go:48 +0x28 fp=0xc007d57c08 sp=0xc007d57bd8 pc=0x73e4e8
net/http/internal.(*chunkedReader).Read(0xc001758540, {0xc00778b000?, 0xc0112e1020?, 0xfeb371c0c18?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/internal/chunked.go:98 +0x156 fp=0xc007d57c88 sp=0xc007d57c08 pc=0x73e7b6
net/http.(*body).readLocked(0xc007647400, {0xc00778b000?, 0xc007850288?, 0xe?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transfer.go:839 +0x3c fp=0xc007d57cd8 sp=0xc007d57c88 pc=0x79695c
net/http.(*body).Read(0xc008ea9b78?, {0xc00778b000?, 0xc0076f6d80?, 0xc037063440?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transfer.go:831 +0x125 fp=0xc007d57d50 sp=0xc007d57cd8 pc=0x796825
net/http.(*bodyEOFSignal).Read(0xc007647440, {0xc00778b000, 0x1000, 0x1000})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2792 +0x142 fp=0xc007d57dd0 sp=0xc007d57d50 pc=0x7a61c2
bufio.(*Reader).fill(0xc00749f500)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/bufio/bufio.go:106 +0xff fp=0xc007d57e08 sp=0xc007d57dd0 pc=0x5389ff
bufio.(*Reader).ReadSlice(0xc00749f500, 0xc0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/bufio/bufio.go:372 +0x2f fp=0xc007d57e58 sp=0xc007d57e08 pc=0x5395ef
bufio.(*Reader).collectFragments(0xc0299407a0?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/bufio/bufio.go:447 +0x74 fp=0xc007d57f18 sp=0xc007d57e58 pc=0x539a34
bufio.(*Reader).ReadBytes(0x1201940?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/bufio/bufio.go:474 +0x1d fp=0xc007d57f98 sp=0xc007d57f18 pc=0x539c5d
github.com/couchbase/indexing/secondary/dcp.(*Client).runObserveStreamingEndpoint.func2()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/dcp/pools.go:482 +0x4d fp=0xc007d57fe0 sp=0xc007d57f98 pc=0x864a2d
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc007d57fe8 sp=0xc007d57fe0 pc=0x473161
created by github.com/couchbase/indexing/secondary/dcp.(*Client).runObserveStreamingEndpoint
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/dcp/pools.go:480 +0x6c5

goroutine 80 [select, 60 minutes]:
runtime.gopark(0xc00002df68?, 0x4?, 0x3?, 0x0?, 0xc00002ddb0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00002dc00 sp=0xc00002dbe0 pc=0x43fd16
runtime.selectgo(0xc00002df68, 0xc00002dda8, 0xc00033e340?, 0x0, 0x1?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc00002dd40 sp=0xc00002dc00 pc=0x4503be
net/http.(*persistConn).readLoop(0xc000328240)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2227 +0xd85 fp=0xc00002dfc8 sp=0xc00002dd40 pc=0x7a34c5
net/http.(*Transport).dialConn.func5()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x26 fp=0xc00002dfe0 sp=0xc00002dfc8 pc=0x7a1086
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00002dfe8 sp=0xc00002dfe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x16ea

goroutine 963 [GC worker (idle)]:
runtime.gopark(0xfe134aff1f1?, 0x3?, 0xe6?, 0xc5?, 0x413700?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc0005fb750 sp=0xc0005fb730 pc=0x43fd16
runtime.gcBgMarkWorker()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc0005fb7e0 sp=0xc0005fb750 pc=0x421091
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0005fb7e8 sp=0xc0005fb7e0 pc=0x473161
created by runtime.gcBgMarkStartWorkers
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1199 +0x25

goroutine 11 [IO wait]:
runtime.gopark(0x125d3db?, 0xb?, 0x0?, 0x0?, 0xb?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00031d9a0 sp=0xc00031d980 pc=0x43fd16
runtime.netpollblock(0x487505?, 0x40934f?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/netpoll.go:527 +0xf7 fp=0xc00031d9d8 sp=0xc00031d9a0 pc=0x438277
internal/poll.runtime_pollWait(0x7fc9c426ac38, 0x72)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/netpoll.go:306 +0x89 fp=0xc00031d9f8 sp=0xc00031d9d8 pc=0x46d209
internal/poll.(*pollDesc).wait(0xc000040580?, 0xc0003cc000?, 0x0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc00031da20 sp=0xc00031d9f8 pc=0x4aa8f2
internal/poll.(*pollDesc).waitRead(...)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000040580, {0xc0003cc000, 0x1000, 0x1000})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_unix.go:167 +0x299 fp=0xc00031dab8 sp=0xc00031da20 pc=0x4abcd9
net.(*netFD).Read(0xc000040580, {0xc0003cc000?, 0x47ae76?, 0xc00031db40?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/fd_posix.go:55 +0x29 fp=0xc00031db00 sp=0xc00031dab8 pc=0x688e49
net.(*conn).Read(0xc00041c000, {0xc0003cc000?, 0xc0079dec66?, 0x5?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/net.go:183 +0x45 fp=0xc00031db48 sp=0xc00031db00 pc=0x699085
net.(*TCPConn).Read(0xc000134720?, {0xc0003cc000?, 0x0?, 0x67?})
	:1 +0x29 fp=0xc00031db78 sp=0xc00031db48 pc=0x6ab8e9
net/http.(*persistConn).Read(0xc000328240, {0xc0003cc000?, 0xc00010e3c0?, 0xc000349418?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1943 +0x4e fp=0xc00031dbd8 sp=0xc00031db78 pc=0x7a1ace
bufio.(*Reader).fill(0xc0000982a0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/bufio/bufio.go:106 +0xff fp=0xc00031dc10 sp=0xc00031dbd8 pc=0x5389ff
bufio.(*Reader).ReadSlice(0xc0000982a0, 0xb?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/bufio/bufio.go:372 +0x2f fp=0xc00031dc60 sp=0xc00031dc10 pc=0x5395ef
net/http/internal.readChunkLine(0x5b?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/internal/chunked.go:129 +0x25 fp=0xc00031dcb0 sp=0xc00031dc60 pc=0x73ea85
net/http/internal.(*chunkedReader).beginChunk(0xc0003209c0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/internal/chunked.go:48 +0x28 fp=0xc00031dce0 sp=0xc00031dcb0 pc=0x73e4e8
net/http/internal.(*chunkedReader).Read(0xc0003209c0, {0xc0079dec00?, 0xc0001555f0?, 0xc0079db260?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/internal/chunked.go:98 +0x156 fp=0xc00031dd60 sp=0xc00031dce0 pc=0x73e7b6
net/http.(*body).readLocked(0xc00033e340, {0xc0079dec00?, 0xc0000c8450?, 0x11793c0?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transfer.go:839 +0x3c fp=0xc00031ddb0 sp=0xc00031dd60 pc=0x79695c
net/http.(*body).Read(0x0?, {0xc0079dec00?, 0x57b3de?, 0x0?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transfer.go:831 +0x125 fp=0xc00031de28 sp=0xc00031ddb0 pc=0x796825
net/http.(*bodyEOFSignal).Read(0xc00033e380, {0xc0079dec00, 0x600, 0x600})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2792 +0x142 fp=0xc00031dea8 sp=0xc00031de28 pc=0x7a61c2
encoding/json.(*Decoder).refill(0xc0000c83c0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/encoding/json/stream.go:165 +0x188 fp=0xc00031def8 sp=0xc00031dea8 pc=0x58e528
encoding/json.(*Decoder).readValue(0xc0000c83c0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/encoding/json/stream.go:140 +0xbb fp=0xc00031df48 sp=0xc00031def8 pc=0x58e11b
encoding/json.(*Decoder).Decode(0xc0000c83c0, {0x1065120, 0xc0001346e0})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/encoding/json/stream.go:63 +0x78 fp=0xc00031df78 sp=0xc00031df48 pc=0x58dd58
github.com/couchbase/cbauth/metakv.doRunObserveChildren.func1()
	/opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:293 +0xc5 fp=0xc00031dfe0 sp=0xc00031df78 pc=0xb33045
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00031dfe8 sp=0xc00031dfe0 pc=0x473161
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:289 +0x2eb

goroutine 12 [select]:
runtime.gopark(0xc00015dd08?, 0x4?, 0x2?, 0x0?, 0xc00015dc90?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00015db08 sp=0xc00015dae8 pc=0x43fd16
runtime.selectgo(0xc00015dd08, 0xc00015dc88, 0xc009db0b40?, 0x0, 0xc003953201?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc00015dc48 sp=0xc00015db08 pc=0x4503be
github.com/couchbase/gometa/protocol.runWatcher(0xc0092e2640, {0x141e7b0, 0xc00039c240}, {0x142f6f8?, 0xc00039c240}, {0x142d940?, 0x1e50278}, 0xc000194540, 0xc000325200, 0xc000194620, ...)
	/opt/build/goproj/src/github.com/couchbase/gometa/protocol/watcherServer.go:357 +0x3be fp=0xc00015dd60 sp=0xc00015dc48 pc=0xdf119e
github.com/couchbase/gometa/protocol.runOnce(0x3, {0xc00012fdb0, 0xe}, {0x141e7b0, 0xc00039c240}, {0x142f6f8, 0xc00039c240}, {0x142d940, 0x1e50278}, 0xc000194540, ...)
	/opt/build/goproj/src/github.com/couchbase/gometa/protocol/watcherServer.go:213 +0x525 fp=0xc00015de90 sp=0xc00015dd60 pc=0xdf0265
github.com/couchbase/gometa/protocol.RunWatcherServerWithRequest2({0xc00012fdb0, 0xe}, {0x141e7b0, 0xc00039c240}, {0x142f6f8, 0xc00039c240}, {0x142d940, 0x1e50278}, 0xc000194540, 0xc000325200, ...)
	/opt/build/goproj/src/github.com/couchbase/gometa/protocol/watcherServer.go:65 +0x174 fp=0xc00015df68 sp=0xc00015de90 pc=0xdefb94
github.com/couchbase/indexing/secondary/manager/client.(*MetadataProvider).startWatcher.func1()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:4487 +0x9f fp=0xc00015dfe0 sp=0xc00015df68 pc=0xe69ebf
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00015dfe8 sp=0xc00015dfe0 pc=0x473161
created by github.com/couchbase/indexing/secondary/manager/client.(*MetadataProvider).startWatcher
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:4487 +0x1f8

goroutine 104 [select, 59 minutes]:
runtime.gopark(0xc00032ef90?, 0x2?, 0xf8?, 0xed?, 0xc00032ef34?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00032edb0 sp=0xc00032ed90 pc=0x43fd16
runtime.selectgo(0xc00032ef90, 0xc00032ef30, 0xc00007e000?, 0x0, 0xc0091b2090?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc00032eef0 sp=0xc00032edb0 pc=0x4503be
net/http.(*persistConn).writeLoop(0xc0000b6480)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2410 +0xf2 fp=0xc00032efc8 sp=0xc00032eef0 pc=0x7a45b2
net/http.(*Transport).dialConn.func6()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x26 fp=0xc00032efe0 sp=0xc00032efc8 pc=0x7a1026
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00032efe8 sp=0xc00032efe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x173d

goroutine 141 [IO wait]:
runtime.gopark(0x0?, 0xb?, 0x0?, 0x0?, 0x11?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00071c9a0 sp=0xc00071c980 pc=0x43fd16
runtime.netpollblock(0x487505?, 0x40934f?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/netpoll.go:527 +0xf7 fp=0xc00071c9d8 sp=0xc00071c9a0 pc=0x438277
internal/poll.runtime_pollWait(0x7fc9c426a4b8, 0x72)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/netpoll.go:306 +0x89 fp=0xc00071c9f8 sp=0xc00071c9d8 pc=0x46d209
internal/poll.(*pollDesc).wait(0xc000410280?, 0xc0005a4000?, 0x0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc00071ca20 sp=0xc00071c9f8 pc=0x4aa8f2
internal/poll.(*pollDesc).waitRead(...)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000410280, {0xc0005a4000, 0x1000, 0x1000})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_unix.go:167 +0x299 fp=0xc00071cab8 sp=0xc00071ca20 pc=0x4abcd9
net.(*netFD).Read(0xc000410280, {0xc0005a4000?, 0x47ae76?, 0xc00071cb40?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/fd_posix.go:55 +0x29 fp=0xc00071cb00 sp=0xc00071cab8 pc=0x688e49
net.(*conn).Read(0xc000014008, {0xc0005a4000?, 0xc0068c7866?, 0x5?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/net.go:183 +0x45 fp=0xc00071cb48 sp=0xc00071cb00 pc=0x699085
net.(*TCPConn).Read(0xc00041f210?, {0xc0005a4000?, 0x0?, 0x67?})
	:1 +0x29 fp=0xc00071cb78 sp=0xc00071cb48 pc=0x6ab8e9
net/http.(*persistConn).Read(0xc0000b6480, {0xc0005a4000?, 0x0?, 0x0?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1943 +0x4e fp=0xc00071cbd8 sp=0xc00071cb78 pc=0x7a1ace
bufio.(*Reader).fill(0xc000400180)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/bufio/bufio.go:106 +0xff fp=0xc00071cc10 sp=0xc00071cbd8 pc=0x5389ff
bufio.(*Reader).ReadSlice(0xc000400180, 0xb?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/bufio/bufio.go:372 +0x2f fp=0xc00071cc60 sp=0xc00071cc10 pc=0x5395ef
net/http/internal.readChunkLine(0x5b?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/internal/chunked.go:129 +0x25 fp=0xc00071ccb0 sp=0xc00071cc60 pc=0x73ea85
net/http/internal.(*chunkedReader).beginChunk(0xc001a61cb0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/internal/chunked.go:48 +0x28 fp=0xc00071cce0 sp=0xc00071ccb0 pc=0x73e4e8
net/http/internal.(*chunkedReader).Read(0xc001a61cb0, {0xc0068c7800?, 0xc0001555f0?, 0xc0068bd4d0?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/internal/chunked.go:98 +0x156 fp=0xc00071cd60 sp=0xc00071cce0 pc=0x73e7b6
net/http.(*body).readLocked(0xc001a6ff00, {0xc0068c7800?, 0xc001a73e90?, 0x11793c0?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transfer.go:839 +0x3c fp=0xc00071cdb0 sp=0xc00071cd60 pc=0x79695c
net/http.(*body).Read(0x0?, {0xc0068c7800?, 0x57b3de?, 0x0?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transfer.go:831 +0x125 fp=0xc00071ce28 sp=0xc00071cdb0 pc=0x796825
net/http.(*bodyEOFSignal).Read(0xc001a6ff40, {0xc0068c7800, 0x600, 0x600})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2792 +0x142 fp=0xc00071cea8 sp=0xc00071ce28 pc=0x7a61c2
encoding/json.(*Decoder).refill(0xc001a73e00)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/encoding/json/stream.go:165 +0x188 fp=0xc00071cef8 sp=0xc00071cea8 pc=0x58e528
encoding/json.(*Decoder).readValue(0xc001a73e00)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/encoding/json/stream.go:140 +0xbb fp=0xc00071cf48 sp=0xc00071cef8 pc=0x58e11b
encoding/json.(*Decoder).Decode(0xc001a73e00, {0x1065120, 0xc00041f1d0})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/encoding/json/stream.go:63 +0x78 fp=0xc00071cf78 sp=0xc00071cf48 pc=0x58dd58
github.com/couchbase/cbauth/metakv.doRunObserveChildren.func1()
	/opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:293 +0xc5 fp=0xc00071cfe0 sp=0xc00071cf78 pc=0xb33045
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00071cfe8 sp=0xc00071cfe0 pc=0x473161
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:289 +0x2eb

goroutine 131 [select, 59 minutes]:
runtime.gopark(0xc000332f90?, 0x2?, 0xf8?, 0x2d?, 0xc000332f34?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000332db0 sp=0xc000332d90 pc=0x43fd16
runtime.selectgo(0xc000332f90, 0xc000332f30, 0xc000165600?, 0x0, 0xc009167140?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc000332ef0 sp=0xc000332db0 pc=0x4503be
net/http.(*persistConn).writeLoop(0xc0000b6c60)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2410 +0xf2 fp=0xc000332fc8 sp=0xc000332ef0 pc=0x7a45b2
net/http.(*Transport).dialConn.func6()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x26 fp=0xc000332fe0 sp=0xc000332fc8 pc=0x7a1026
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000332fe8 sp=0xc000332fe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x173d

goroutine 5291 [select, 42 minutes]:
runtime.gopark(0xc00555df68?, 0x4?, 0x3?, 0x0?, 0xc00555ddb0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00555dc00 sp=0xc00555dbe0 pc=0x43fd16
runtime.selectgo(0xc00555df68, 0xc00555dda8, 0xc018717540?, 0x0, 0xc03228f001?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc00555dd40 sp=0xc00555dc00 pc=0x4503be
net/http.(*persistConn).readLoop(0xc01a9ea5a0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2227 +0xd85 fp=0xc00555dfc8 sp=0xc00555dd40 pc=0x7a34c5
net/http.(*Transport).dialConn.func5()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x26 fp=0xc00555dfe0 sp=0xc00555dfc8 pc=0x7a1086
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00555dfe8 sp=0xc00555dfe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x16ea

goroutine 5423 [select, 42 minutes]:
runtime.gopark(0xc01e588f68?, 0x4?, 0x65?, 0xa6?, 0xc01e588db0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc01e588c00 sp=0xc01e588be0 pc=0x43fd16
runtime.selectgo(0xc01e588f68, 0xc01e588da8, 0xc01be28a80?, 0x0, 0xc00882cd01?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc01e588d40 sp=0xc01e588c00 pc=0x4503be
net/http.(*persistConn).readLoop(0xc01a9eb200)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2227 +0xd85 fp=0xc01e588fc8 sp=0xc01e588d40 pc=0x7a34c5
net/http.(*Transport).dialConn.func5()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x26 fp=0xc01e588fe0 sp=0xc01e588fc8 pc=0x7a1086
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc01e588fe8 sp=0xc01e588fe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x16ea

goroutine 3158 [IO wait]:
runtime.gopark(0xc004dd3c98?, 0xb?, 0x0?, 0x0?, 0x15?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc004dd3c68 sp=0xc004dd3c48 pc=0x43fd16
runtime.netpollblock(0x487505?, 0x40934f?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/netpoll.go:527 +0xf7 fp=0xc004dd3ca0 sp=0xc004dd3c68 pc=0x438277
internal/poll.runtime_pollWait(0x7fc9c426a2d8, 0x72)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/netpoll.go:306 +0x89 fp=0xc004dd3cc0 sp=0xc004dd3ca0 pc=0x46d209
internal/poll.(*pollDesc).wait(0xc0091d8b80?, 0xc002edc190?, 0x0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc004dd3ce8 sp=0xc004dd3cc0 pc=0x4aa8f2
internal/poll.(*pollDesc).waitRead(...)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0091d8b80, {0xc002edc190, 0x8, 0x8})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_unix.go:167 +0x299 fp=0xc004dd3d80 sp=0xc004dd3ce8 pc=0x4abcd9
net.(*netFD).Read(0xc0091d8b80, {0xc002edc190?, 0x7fc9bcaf0538?, 0x7fc9ec641108?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/fd_posix.go:55 +0x29 fp=0xc004dd3dc8 sp=0xc004dd3d80 pc=0x688e49
net.(*conn).Read(0xc00b1d4518, {0xc002edc190?, 0x1095980?, 0xc001059501?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/net.go:183 +0x45 fp=0xc004dd3e10 sp=0xc004dd3dc8 pc=0x699085
net.(*TCPConn).Read(0xd38b1e?, {0xc002edc190?, 0xc004dd3ed8?, 0xd381af?})
	:1 +0x29 fp=0xc004dd3e40 sp=0xc004dd3e10 pc=0x6ab8e9
github.com/couchbase/gometa/common.(*PeerPipe).readBytes(0xc008459e00, 0x1427340?, {0x0?, 0xc02cde6f80?, 0x174876e800?})
	/opt/build/goproj/src/github.com/couchbase/gometa/common/peerPipe.go:313 +0xcf fp=0xc004dd3ee8 sp=0xc004dd3e40 pc=0xd382ef
github.com/couchbase/gometa/common.(*PeerPipe).doReceive(0xc008459e00)
	/opt/build/goproj/src/github.com/couchbase/gometa/common/peerPipe.go:214 +0xd5 fp=0xc004dd3fc8 sp=0xc004dd3ee8 pc=0xd37495
github.com/couchbase/gometa/common.NewPeerPipe.func2()
	/opt/build/goproj/src/github.com/couchbase/gometa/common/peerPipe.go:66 +0x26 fp=0xc004dd3fe0 sp=0xc004dd3fc8 pc=0xd36546
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc004dd3fe8 sp=0xc004dd3fe0 pc=0x473161
created by github.com/couchbase/gometa/common.NewPeerPipe
	/opt/build/goproj/src/github.com/couchbase/gometa/common/peerPipe.go:66 +0x18a

goroutine 960 [select, 54 minutes]:
runtime.gopark(0xc00031cf68?, 0x4?, 0x3?, 0x0?, 0xc00031cdb0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00031cc00 sp=0xc00031cbe0 pc=0x43fd16
runtime.selectgo(0xc00031cf68, 0xc00031cda8, 0xc00ac00c40?, 0x0, 0x1?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc00031cd40 sp=0xc00031cc00 pc=0x4503be
net/http.(*persistConn).readLoop(0xc0074259e0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2227 +0xd85 fp=0xc00031cfc8 sp=0xc00031cd40 pc=0x7a34c5
net/http.(*Transport).dialConn.func5()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x26 fp=0xc00031cfe0 sp=0xc00031cfc8 pc=0x7a1086
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00031cfe8 sp=0xc00031cfe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x16ea

goroutine 16 [chan send, 60 minutes]:
runtime.gopark(0xc75b68?, 0xc0068c8000?, 0x2e?, 0x19?, 0x12a99b9?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000335d60 sp=0xc000335d40 pc=0x43fd16
runtime.chansend(0xc000325440, 0x1411698, 0x1, 0xc00055bd01?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/chan.go:259 +0x42e fp=0xc000335de8 sp=0xc000335d60 pc=0x40bc6e
runtime.chansend1(0xc0001391d0?, 0x2c?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/chan.go:145 +0x1d fp=0xc000335e18 sp=0xc000335de8 pc=0x40b81d
github.com/couchbase/indexing/secondary/common.(*internalVersionMonitor).notifier(0xc0000c4230)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/common/internal_version.go:745 +0x465 fp=0xc000335fc8 sp=0xc000335e18 pc=0xc762c5
github.com/couchbase/indexing/secondary/common.newInternalVersionMonitor.func2()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/common/internal_version.go:238 +0x26 fp=0xc000335fe0 sp=0xc000335fc8 pc=0xc717e6
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000335fe8 sp=0xc000335fe0 pc=0x473161
created by github.com/couchbase/indexing/secondary/common.newInternalVersionMonitor
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/common/internal_version.go:238 +0x3e5

goroutine 101 [chan receive]:
runtime.gopark(0xc00031e1e0?, 0xc008df03a8?, 0x20?, 0x4c?, 0xc0000580e0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc0068daec8 sp=0xc0068daea8 pc=0x43fd16
runtime.chanrecv(0xc000400c00, 0x0, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/chan.go:583 +0x49d fp=0xc0068daf58 sp=0xc0068daec8 pc=0x40ca7d
runtime.chanrecv1(0xc0000f0210?, 0xc0005fef47?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/chan.go:442 +0x18 fp=0xc0068daf80 sp=0xc0068daf58 pc=0x40c578
github.com/couchbase/indexing/secondary/queryport/client.(*metadataClient).logstats(0xc0000f0210)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:1410 +0x79 fp=0xc0068dafc8 sp=0xc0068daf80 pc=0xea1f59
github.com/couchbase/indexing/secondary/queryport/client.newMetaBridgeClient.func2()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:152 +0x26 fp=0xc0068dafe0 sp=0xc0068dafc8 pc=0xe9a9e6
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0068dafe8 sp=0xc0068dafe0 pc=0x473161
created by github.com/couchbase/indexing/secondary/queryport/client.newMetaBridgeClient
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:152 +0x616

goroutine 1014 [GC worker (idle)]:
runtime.gopark(0xfe1348dda24?, 0x3?, 0x71?, 0xc9?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00999df50 sp=0xc00999df30 pc=0x43fd16
runtime.gcBgMarkWorker()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc00999dfe0 sp=0xc00999df50 pc=0x421091
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00999dfe8 sp=0xc00999dfe0 pc=0x473161
created by runtime.gcBgMarkStartWorkers
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1199 +0x25

goroutine 927 [GC worker (idle)]:
runtime.gopark(0xfe12ab2748e?, 0x1?, 0x74?, 0x14?, 0xc00a1965a0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000349750 sp=0xc000349730 pc=0x43fd16
runtime.gcBgMarkWorker()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc0003497e0 sp=0xc000349750 pc=0x421091
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0003497e8 sp=0xc0003497e0 pc=0x473161
created by runtime.gcBgMarkStartWorkers
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1199 +0x25

goroutine 925 [GC worker (idle)]:
runtime.gopark(0xfe134dfcd6e?, 0x1?, 0xab?, 0xa7?, 0xc00a196660?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc0005fff50 sp=0xc0005fff30 pc=0x43fd16
runtime.gcBgMarkWorker()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc0005fffe0 sp=0xc0005fff50 pc=0x421091
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0005fffe8 sp=0xc0005fffe0 pc=0x473161
created by runtime.gcBgMarkStartWorkers
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1199 +0x25

goroutine 1013 [GC worker (idle)]:
runtime.gopark(0x1e515a0?, 0x1?, 0x2a?, 0xbb?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00999d750 sp=0xc00999d730 pc=0x43fd16
runtime.gcBgMarkWorker()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc00999d7e0 sp=0xc00999d750 pc=0x421091
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00999d7e8 sp=0xc00999d7e0 pc=0x473161
created by runtime.gcBgMarkStartWorkers
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1199 +0x25

goroutine 1010 [GC worker (idle)]:
runtime.gopark(0x1e515a0?, 0x1?, 0x96?, 0x69?, 0x98?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00999c750 sp=0xc00999c730 pc=0x43fd16
runtime.gcBgMarkWorker()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc00999c7e0 sp=0xc00999c750 pc=0x421091
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00999c7e8 sp=0xc00999c7e0 pc=0x473161
created by runtime.gcBgMarkStartWorkers
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1199 +0x25

goroutine 105 [select]:
runtime.gopark(0xc00071ff90?, 0x2?, 0x0?, 0x0?, 0xc00071ff8c?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00071fe18 sp=0xc00071fdf8 pc=0x43fd16
runtime.selectgo(0xc00071ff90, 0xc00071ff88, 0x0?, 0x0, 0xc000165400?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc00071ff58 sp=0xc00071fe18 pc=0x4503be
github.com/couchbase/indexing/secondary/queryport/client.(*GsiClient).listenMetaChange(0xc000124280, 0xc0000f4150)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/client.go:1830 +0x6e fp=0xc00071ffc0 sp=0xc00071ff58 pc=0xe95c6e
github.com/couchbase/indexing/secondary/queryport/client.makeWithMetaProvider.func1()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/client.go:1823 +0x2a fp=0xc00071ffe0 sp=0xc00071ffc0 pc=0xe95bca
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00071ffe8 sp=0xc00071ffe0 pc=0x473161
created by github.com/couchbase/indexing/secondary/queryport/client.makeWithMetaProvider
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/client.go:1823 +0x298

goroutine 103 [select, 59 minutes]:
runtime.gopark(0xc00002cf68?, 0x4?, 0x3?, 0x0?, 0xc00002cdb0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00002cc00 sp=0xc00002cbe0 pc=0x43fd16
runtime.selectgo(0xc00002cf68, 0xc00002cda8, 0xc001a6ff00?, 0x0, 0x1?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc00002cd40 sp=0xc00002cc00 pc=0x4503be
net/http.(*persistConn).readLoop(0xc0000b6480)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2227 +0xd85 fp=0xc00002cfc8 sp=0xc00002cd40 pc=0x7a34c5
net/http.(*Transport).dialConn.func5()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x26 fp=0xc00002cfe0 sp=0xc00002cfc8 pc=0x7a1086
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00002cfe8 sp=0xc00002cfe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x16ea

goroutine 127 [sleep]:
runtime.gopark(0xfede533c1e3?, 0xc000720f18?, 0xc5?, 0xcf?, 0x10?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000720ee8 sp=0xc000720ec8 pc=0x43fd16
time.Sleep(0x3b9aca00)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/time.go:195 +0x135 fp=0xc000720f28 sp=0xc000720ee8 pc=0x46ffd5
github.com/couchbase/indexing/secondary/queryport/client.(*connectionPool).releaseConnsRoutine(0xc0000f0f20)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/conn_pool.go:461 +0xa5 fp=0xc000720fc8 sp=0xc000720f28 pc=0xe9a105
github.com/couchbase/indexing/secondary/queryport/client.newConnectionPool.func1()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/conn_pool.go:101 +0x26 fp=0xc000720fe0 sp=0xc000720fc8 pc=0xe97366
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000720fe8 sp=0xc000720fe0 pc=0x473161
created by github.com/couchbase/indexing/secondary/queryport/client.newConnectionPool
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/conn_pool.go:101 +0x4b6

goroutine 106 [select]:
runtime.gopark(0xc000317f50?, 0x2?, 0xbc?, 0xfe?, 0xc000317f14?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000317d90 sp=0xc000317d70 pc=0x43fd16
runtime.selectgo(0xc000317f50, 0xc000317f10, 0x12803a6?, 0x0, 0xc000317f20?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc000317ed0 sp=0xc000317d90 pc=0x4503be
github.com/couchbase/indexing/secondary/queryport/client.(*GsiClient).logstats(0xc000124280, 0xc0000f4150)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/client.go:1849 +0x238 fp=0xc000317fc0 sp=0xc000317ed0 pc=0xe95ed8
github.com/couchbase/indexing/secondary/queryport/client.makeWithMetaProvider.func2()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/client.go:1824 +0x2a fp=0xc000317fe0 sp=0xc000317fc0 pc=0xe95b6a
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000317fe8 sp=0xc000317fe0 pc=0x473161
created by github.com/couchbase/indexing/secondary/queryport/client.makeWithMetaProvider
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/client.go:1824 +0x2fa

goroutine 139 [select, 54 minutes]:
runtime.gopark(0xc008039bc0?, 0x3?, 0x0?, 0x0?, 0xc008039b3a?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc007d55998 sp=0xc007d55978 pc=0x43fd16
runtime.selectgo(0xc007d55bc0, 0xc008039b34, 0xc007dd0000?, 0x0, 0xf12?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc007d55ad8 sp=0xc007d55998 pc=0x4503be
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc000196620?, {0x127007b, 0x13}, 0xc007d55e88, 0xc0007240c0)
	/opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:310 +0x429 fp=0xc007d55e60 sp=0xc007d55ad8 pc=0xb32c09
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(...)
	/opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:268
github.com/couchbase/cbauth/metakv.RunObserveChildren({0x127007b?, 0x1d91380?}, 0xc00041c930?, 0xc0000481d0?)
	/opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:398 +0x58 fp=0xc007d55ea8 sp=0xc007d55e60 pc=0xb33158
github.com/couchbase/indexing/secondary/queryport/client.NewClientSettings.func1.1(0x1064c20?, {0x0?, 0x0?})
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/settings.go:90 +0xcc fp=0xc007d55f28 sp=0xc007d55ea8 pc=0xec3ccc
github.com/couchbase/indexing/secondary/common.(*RetryHelper).Run(0xc007d55fa8)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/common/retry_helper.go:36 +0x83 fp=0xc007d55f60 sp=0xc007d55f28 pc=0xc79063
github.com/couchbase/indexing/secondary/queryport/client.NewClientSettings.func1()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/settings.go:94 +0x75 fp=0xc007d55fe0 sp=0xc007d55f60 pc=0xec3bb5
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc007d55fe8 sp=0xc007d55fe0 pc=0x473161
created by github.com/couchbase/indexing/secondary/queryport/client.NewClientSettings
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/settings.go:85 +0x1cf

goroutine 5310 [select, 42 minutes]:
runtime.gopark(0xc031545f90?, 0x2?, 0xf8?, 0x5d?, 0xc031545f34?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc031545db0 sp=0xc031545d90 pc=0x43fd16
runtime.selectgo(0xc031545f90, 0xc031545f30, 0xc0188e3b80?, 0x0, 0xc01bc14270?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc031545ef0 sp=0xc031545db0 pc=0x4503be
net/http.(*persistConn).writeLoop(0xc01b9dc240)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2410 +0xf2 fp=0xc031545fc8 sp=0xc031545ef0 pc=0x7a45b2
net/http.(*Transport).dialConn.func6()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x26 fp=0xc031545fe0 sp=0xc031545fc8 pc=0x7a1026
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc031545fe8 sp=0xc031545fe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x173d

goroutine 147 [select]:
runtime.gopark(0xc000027ba0?, 0x3?, 0x8a?, 0x37?, 0xc000027b1a?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000027978 sp=0xc000027958 pc=0x43fd16
runtime.selectgo(0xc000027ba0, 0xc000027b14, 0x0?, 0x0, 0x0?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc000027ab8 sp=0xc000027978 pc=0x4503be
github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc000196620?, {0x127bf71, 0x1b}, 0xc000027e68, 0xc000324180)
	/opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:310 +0x429 fp=0xc000027e40 sp=0xc000027ab8 pc=0xb32c09
github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(...)
	/opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:268
github.com/couchbase/cbauth/metakv.RunObserveChildren({0x127bf71?, 0xc?}, 0xffffffffffffffff?, 0x7300630000000000?)
	/opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:398 +0x58 fp=0xc000027e88 sp=0xc000027e40 pc=0xb33158
github.com/couchbase/indexing/secondary/manager/common.(*CommandListener).ListenTokens.func2.1(0xc007657710?, {0x0?, 0x0?})
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/manager/common/token.go:1626 +0xc7 fp=0xc000027f00 sp=0xc000027e88 pc=0xd3e8c7
github.com/couchbase/indexing/secondary/common.(*RetryHelper).Run(0xc000027fa0)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/common/retry_helper.go:36 +0x83 fp=0xc000027f38 sp=0xc000027f00 pc=0xc79063
github.com/couchbase/indexing/secondary/manager/common.(*CommandListener).ListenTokens.func2()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/manager/common/token.go:1631 +0xdf fp=0xc000027fe0 sp=0xc000027f38 pc=0xd3e77f
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000027fe8 sp=0xc000027fe0 pc=0x473161
created by github.com/couchbase/indexing/secondary/manager/common.(*CommandListener).ListenTokens
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/manager/common/token.go:1619 +0xaf

goroutine 150 [IO wait, 54 minutes]:
runtime.gopark(0x0?, 0xb?, 0x0?, 0x0?, 0xe?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc0000839a0 sp=0xc000083980 pc=0x43fd16
runtime.netpollblock(0x487505?, 0x40934f?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/netpoll.go:527 +0xf7 fp=0xc0000839d8 sp=0xc0000839a0 pc=0x438277
internal/poll.runtime_pollWait(0x7fc9c426a788, 0x72)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/netpoll.go:306 +0x89 fp=0xc0000839f8 sp=0xc0000839d8 pc=0x46d209
internal/poll.(*pollDesc).wait(0xc000411400?, 0xc0005a5000?, 0x0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 fp=0xc000083a20 sp=0xc0000839f8 pc=0x4aa8f2
internal/poll.(*pollDesc).waitRead(...)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000411400, {0xc0005a5000, 0x1000, 0x1000})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/internal/poll/fd_unix.go:167 +0x299 fp=0xc000083ab8 sp=0xc000083a20 pc=0x4abcd9
net.(*netFD).Read(0xc000411400, {0xc0005a5000?, 0x47ae76?, 0xc000083b40?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/fd_posix.go:55 +0x29 fp=0xc000083b00 sp=0xc000083ab8 pc=0x688e49
net.(*conn).Read(0xc00041c488, {0xc0005a5000?, 0xc0054e14f4?, 0x5?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/net.go:183 +0x45 fp=0xc000083b48 sp=0xc000083b00 pc=0x699085
net.(*TCPConn).Read(0xc0000a25e0?, {0xc0005a5000?, 0x0?, 0x14f5?})
	:1 +0x29 fp=0xc000083b78 sp=0xc000083b48 pc=0x6ab8e9
net/http.(*persistConn).Read(0xc0000b6c60, {0xc0005a5000?, 0x7fc9ec641108?, 0x3a?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1943 +0x4e fp=0xc000083bd8 sp=0xc000083b78 pc=0x7a1ace
bufio.(*Reader).fill(0xc000099740)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/bufio/bufio.go:106 +0xff fp=0xc000083c10 sp=0xc000083bd8 pc=0x5389ff
bufio.(*Reader).ReadSlice(0xc000099740, 0xb?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/bufio/bufio.go:372 +0x2f fp=0xc000083c60 sp=0xc000083c10 pc=0x5395ef
net/http/internal.readChunkLine(0x14e9?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/internal/chunked.go:129 +0x25 fp=0xc000083cb0 sp=0xc000083c60 pc=0x73ea85
net/http/internal.(*chunkedReader).beginChunk(0xc005385b00)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/internal/chunked.go:48 +0x28 fp=0xc000083ce0 sp=0xc000083cb0 pc=0x73e4e8
net/http/internal.(*chunkedReader).Read(0xc005385b00, {0xc0054e0000?, 0xc0001555f0?, 0xc005385b30?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/internal/chunked.go:98 +0x156 fp=0xc000083d60 sp=0xc000083ce0 pc=0x73e7b6
net/http.(*body).readLocked(0xc005398bc0, {0xc0054e0000?, 0xc0054dc090?, 0x11793c0?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transfer.go:839 +0x3c fp=0xc000083db0 sp=0xc000083d60 pc=0x79695c
net/http.(*body).Read(0x0?, {0xc0054e0000?, 0x57b3de?, 0x0?})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transfer.go:831 +0x125 fp=0xc000083e28 sp=0xc000083db0 pc=0x796825
net/http.(*bodyEOFSignal).Read(0xc005398c00, {0xc0054e0000, 0x1e00, 0x1e00})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2792 +0x142 fp=0xc000083ea8 sp=0xc000083e28 pc=0x7a61c2
encoding/json.(*Decoder).refill(0xc0054dc000)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/encoding/json/stream.go:165 +0x188 fp=0xc000083ef8 sp=0xc000083ea8 pc=0x58e528
encoding/json.(*Decoder).readValue(0xc0054dc000)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/encoding/json/stream.go:140 +0xbb fp=0xc000083f48 sp=0xc000083ef8 pc=0x58e11b
encoding/json.(*Decoder).Decode(0xc0054dc000, {0x1065120, 0xc0000a25a0})
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/encoding/json/stream.go:63 +0x78 fp=0xc000083f78 sp=0xc000083f48 pc=0x58dd58
github.com/couchbase/cbauth/metakv.doRunObserveChildren.func1()
	/opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:293 +0xc5 fp=0xc000083fe0 sp=0xc000083f78 pc=0xb33045
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000083fe8 sp=0xc000083fe0 pc=0x473161
created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
	/opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:289 +0x2eb

goroutine 163 [sleep]:
runtime.gopark(0xfedf5cbe7fa?, 0xc009adef18?, 0xc5?, 0xcf?, 0x10?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc009adeee8 sp=0xc009adeec8 pc=0x43fd16
time.Sleep(0x3b9aca00)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/time.go:195 +0x135 fp=0xc009adef28 sp=0xc009adeee8 pc=0x46ffd5
github.com/couchbase/indexing/secondary/queryport/client.(*connectionPool).releaseConnsRoutine(0xc000465b80)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/conn_pool.go:461 +0xa5 fp=0xc009adefc8 sp=0xc009adef28 pc=0xe9a105
github.com/couchbase/indexing/secondary/queryport/client.newConnectionPool.func1()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/conn_pool.go:101 +0x26 fp=0xc009adefe0 sp=0xc009adefc8 pc=0xe97366
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc009adefe8 sp=0xc009adefe0 pc=0x473161
created by github.com/couchbase/indexing/secondary/queryport/client.newConnectionPool
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/conn_pool.go:101 +0x4b6

goroutine 153 [select, 49 minutes]:
runtime.gopark(0xc000555d08?, 0x4?, 0x0?, 0x0?, 0xc000555c90?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000555b08 sp=0xc000555ae8 pc=0x43fd16
runtime.selectgo(0xc000555d08, 0xc000555c88, 0xc000555c70?, 0x0, 0xc003b4b601?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc000555c48 sp=0xc000555b08 pc=0x4503be
github.com/couchbase/gometa/protocol.runWatcher(0xc008459e00, {0x141e7b0, 0xc0006aa240}, {0x142f6f8?, 0xc0006aa240}, {0x142d940?, 0x1e50278}, 0xc0000f4d90, 0xc00010f0e0, 0xc0000f4e00, ...)
	/opt/build/goproj/src/github.com/couchbase/gometa/protocol/watcherServer.go:357 +0x3be fp=0xc000555d60 sp=0xc000555c48 pc=0xdf119e
github.com/couchbase/gometa/protocol.runOnce(0x3, {0xc001a7fe20, 0xe}, {0x141e7b0, 0xc0006aa240}, {0x142f6f8, 0xc0006aa240}, {0x142d940, 0x1e50278}, 0xc0000f4d90, ...)
	/opt/build/goproj/src/github.com/couchbase/gometa/protocol/watcherServer.go:213 +0x525 fp=0xc000555e90 sp=0xc000555d60 pc=0xdf0265
github.com/couchbase/gometa/protocol.RunWatcherServerWithRequest2({0xc001a7fe20, 0xe}, {0x141e7b0, 0xc0006aa240}, {0x142f6f8, 0xc0006aa240}, {0x142d940, 0x1e50278}, 0xc0000f4d90, 0xc00010f0e0, ...)
	/opt/build/goproj/src/github.com/couchbase/gometa/protocol/watcherServer.go:65 +0x174 fp=0xc000555f68 sp=0xc000555e90 pc=0xdefb94
github.com/couchbase/indexing/secondary/manager/client.(*MetadataProvider).startWatcher.func1()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:4487 +0x9f fp=0xc000555fe0 sp=0xc000555f68 pc=0xe69ebf
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000555fe8 sp=0xc000555fe0 pc=0x473161
created by github.com/couchbase/indexing/secondary/manager/client.(*MetadataProvider).startWatcher
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:4487 +0x1f8

goroutine 1168 [sync.Cond.Wait]:
runtime.gopark(0xc009ce66c0?, 0x19?, 0x0?, 0x0?, 0xc009ae5da8?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc009ae5d78 sp=0xc009ae5d58 pc=0x43fd16
runtime.goparkunlock(...)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:387
sync.runtime_notifyListWait(0xc00645a010, 0x0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/sema.go:527 +0x14c fp=0xc009ae5dc0 sp=0xc009ae5d78 pc=0x46f12c
sync.(*Cond).Wait(0xc032880000?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/sync/cond.go:70 +0x8c fp=0xc009ae5df8 sp=0xc009ae5dc0 pc=0x47ac4c
gopkg.in/couchbase/gocbcore%2ev7.(*memdOpQueue).pop(0xc00a292960, 0xc042aa2010)
	/opt/build/go/pkg/mod/gopkg.in/couchbase/gocbcore.v7@v7.1.18/memdopqueue.go:138 +0x54 fp=0xc009ae5e48 sp=0xc009ae5df8 pc=0xd0e4b4
gopkg.in/couchbase/gocbcore%2ev7.(*memdOpConsumer).Pop(...)
	/opt/build/go/pkg/mod/gopkg.in/couchbase/gocbcore.v7@v7.1.18/memdopqueue.go:28
gopkg.in/couchbase/gocbcore%2ev7.(*memdPipelineClient).ioLoop(0xc002deda80, 0xc009ddb900)
	/opt/build/go/pkg/mod/gopkg.in/couchbase/gocbcore.v7@v7.1.18/memdpipelineclient.go:129 +0x505 fp=0xc009ae5f48 sp=0xc009ae5e48 pc=0xd0fcc5
gopkg.in/couchbase/gocbcore%2ev7.(*memdPipelineClient).Run(0xc002deda80)
	/opt/build/go/pkg/mod/gopkg.in/couchbase/gocbcore.v7@v7.1.18/memdpipelineclient.go:195 +0x346 fp=0xc009ae5fc8 sp=0xc009ae5f48 pc=0xd109e6
gopkg.in/couchbase/gocbcore%2ev7.(*memdPipeline).StartClients.func2()
	/opt/build/go/pkg/mod/gopkg.in/couchbase/gocbcore.v7@v7.1.18/memdpipeline.go:70 +0x26 fp=0xc009ae5fe0 sp=0xc009ae5fc8 pc=0xd0eda6
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc009ae5fe8 sp=0xc009ae5fe0 pc=0x473161
created by gopkg.in/couchbase/gocbcore%2ev7.(*memdPipeline).StartClients
	/opt/build/go/pkg/mod/gopkg.in/couchbase/gocbcore.v7@v7.1.18/memdpipeline.go:70 +0x91

goroutine 1033 [GC worker (idle)]:
runtime.gopark(0x1e515a0?, 0x1?, 0x0?, 0xfa?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00aaf8f50 sp=0xc00aaf8f30 pc=0x43fd16
runtime.gcBgMarkWorker()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc00aaf8fe0 sp=0xc00aaf8f50 pc=0x421091
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00aaf8fe8 sp=0xc00aaf8fe0 pc=0x473161
created by runtime.gcBgMarkStartWorkers
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1199 +0x25

goroutine 179 [select]:
runtime.gopark(0xc000348790?, 0x2?, 0x4f?, 0x9d?, 0xc000348774?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc0003485f8 sp=0xc0003485d8 pc=0x43fd16
runtime.selectgo(0xc000348790, 0xc000348770, 0x0?, 0x0, 0x0?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc000348738 sp=0xc0003485f8 pc=0x4503be
github.com/couchbase/indexing/secondary/manager/client.(*watcher).timeoutChecker(0xc0006aa240)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:6686 +0xcd fp=0xc0003487c8 sp=0xc000348738 pc=0xe781ad
github.com/couchbase/indexing/secondary/manager/client.(*watcher).startTimer.func1()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:6677 +0x26 fp=0xc0003487e0 sp=0xc0003487c8 pc=0xe780a6
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0003487e8 sp=0xc0003487e0 pc=0x473161
created by github.com/couchbase/indexing/secondary/manager/client.(*watcher).startTimer
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:6677 +0x8d

goroutine 158 [chan receive]:
runtime.gopark(0xc005610360?, 0xc007cec000?, 0x20?, 0x4c?, 0xc0000580e0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00032fec8 sp=0xc00032fea8 pc=0x43fd16
runtime.chanrecv(0xc005610300, 0x0, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/chan.go:583 +0x49d fp=0xc00032ff58 sp=0xc00032fec8 pc=0x40ca7d
runtime.chanrecv1(0xc00070e160?, 0x12611e5?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/chan.go:442 +0x18 fp=0xc00032ff80 sp=0xc00032ff58 pc=0x40c578
github.com/couchbase/indexing/secondary/queryport/client.(*metadataClient).logstats(0xc00070e160)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:1410 +0x79 fp=0xc00032ffc8 sp=0xc00032ff80 pc=0xea1f59
github.com/couchbase/indexing/secondary/queryport/client.newMetaBridgeClient.func2()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:152 +0x26 fp=0xc00032ffe0 sp=0xc00032ffc8 pc=0xe9a9e6
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00032ffe8 sp=0xc00032ffe0 pc=0x473161
created by github.com/couchbase/indexing/secondary/queryport/client.newMetaBridgeClient
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:152 +0x616

goroutine 159 [select]:
runtime.gopark(0xc0079eef90?, 0x2?, 0x0?, 0x0?, 0xc0079eef8c?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc0079eee18 sp=0xc0079eedf8 pc=0x43fd16
runtime.selectgo(0xc0079eef90, 0xc0079eef88, 0x0?, 0x0, 0x0?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc0079eef58 sp=0xc0079eee18 pc=0x4503be
github.com/couchbase/indexing/secondary/queryport/client.(*GsiClient).listenMetaChange(0xc00a116480, 0xc0004c00e0)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/client.go:1830 +0x6e fp=0xc0079eefc0 sp=0xc0079eef58 pc=0xe95c6e
github.com/couchbase/indexing/secondary/queryport/client.makeWithMetaProvider.func1()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/client.go:1823 +0x2a fp=0xc0079eefe0 sp=0xc0079eefc0 pc=0xe95bca
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0079eefe8 sp=0xc0079eefe0 pc=0x473161
created by github.com/couchbase/indexing/secondary/queryport/client.makeWithMetaProvider
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/client.go:1823 +0x298

goroutine 160 [select]:
runtime.gopark(0xc000082f50?, 0x2?, 0x3b?, 0xad?, 0xc000082f14?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000082d90 sp=0xc000082d70 pc=0x43fd16
runtime.selectgo(0xc000082f50, 0xc000082f10, 0x12803a6?, 0x0, 0xc000082f20?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc000082ed0 sp=0xc000082d90 pc=0x4503be
github.com/couchbase/indexing/secondary/queryport/client.(*GsiClient).logstats(0xc00a116480, 0xc0004c00e0)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/client.go:1849 +0x238 fp=0xc000082fc0 sp=0xc000082ed0 pc=0xe95ed8
github.com/couchbase/indexing/secondary/queryport/client.makeWithMetaProvider.func2()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/client.go:1824 +0x2a fp=0xc000082fe0 sp=0xc000082fc0 pc=0xe95b6a
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000082fe8 sp=0xc000082fe0 pc=0x473161
created by github.com/couchbase/indexing/secondary/queryport/client.makeWithMetaProvider
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/client.go:1824 +0x2fa

goroutine 5292 [select, 42 minutes]:
runtime.gopark(0xc00555ff90?, 0x2?, 0xf8?, 0xfd?, 0xc00555ff34?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00555fdb0 sp=0xc00555fd90 pc=0x43fd16
runtime.selectgo(0xc00555ff90, 0xc00555ff30, 0xc01a82b6c0?, 0x0, 0xc01add39b0?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc00555fef0 sp=0xc00555fdb0 pc=0x4503be
net/http.(*persistConn).writeLoop(0xc01a9ea5a0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2410 +0xf2 fp=0xc00555ffc8 sp=0xc00555fef0 pc=0x7a45b2
net/http.(*Transport).dialConn.func6()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x26 fp=0xc00555ffe0 sp=0xc00555ffc8 pc=0x7a1026
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00555ffe8 sp=0xc00555ffe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x173d

goroutine 195 [chan receive]:
runtime.gopark(0x0?, 0x0?, 0x13?, 0x0?, 0xc000080e50?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000080da8 sp=0xc000080d88 pc=0x43fd16
runtime.chanrecv(0xc00539ee40, 0x0, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/chan.go:583 +0x49d fp=0xc000080e38 sp=0xc000080da8 pc=0x40ca7d
runtime.chanrecv1(0xc000080f48?, 0x13?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/chan.go:442 +0x18 fp=0xc000080e60 sp=0xc000080e38 pc=0x40c578
github.com/couchbase/indexing/secondary/queryport/n1ql.(*monitor).backfillMonitor(0xc0068b4a20, 0x0?)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/n1ql/monitor.go:91 +0x9b fp=0xc000080fc0 sp=0xc000080e60 pc=0xed1dbb
github.com/couchbase/indexing/secondary/queryport/n1ql.initGSIMonitor.func1()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/n1ql/monitor.go:40 +0x2a fp=0xc000080fe0 sp=0xc000080fc0 pc=0xed166a
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000080fe8 sp=0xc000080fe0 pc=0x473161
created by github.com/couchbase/indexing/secondary/queryport/n1ql.initGSIMonitor
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/n1ql/monitor.go:40 +0xb8

goroutine 196 [chan receive]:
runtime.gopark(0xc000401200?, 0x7fc9277d4778?, 0x5?, 0x0?, 0xc0050f2de8?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc0050f2d68 sp=0xc0050f2d48 pc=0x43fd16
runtime.chanrecv(0xc00539eea0, 0x0, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/chan.go:583 +0x49d fp=0xc0050f2df8 sp=0xc0050f2d68 pc=0x40ca7d
runtime.chanrecv1(0xc0050f2eb8?, 0x7?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/chan.go:442 +0x18 fp=0xc0050f2e20 sp=0xc0050f2df8 pc=0x40c578
github.com/couchbase/indexing/secondary/queryport/n1ql.(*monitor).logStats(0xc0068b4a20, 0x0?)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/n1ql/monitor.go:124 +0x9b fp=0xc0050f2fc0 sp=0xc0050f2e20 pc=0xed217b
github.com/couchbase/indexing/secondary/queryport/n1ql.initGSIMonitor.func2()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/n1ql/monitor.go:41 +0x2a fp=0xc0050f2fe0 sp=0xc0050f2fc0 pc=0xed160a
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0050f2fe8 sp=0xc0050f2fe0 pc=0x473161
created by github.com/couchbase/indexing/secondary/queryport/n1ql.initGSIMonitor
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/queryport/n1ql/monitor.go:41 +0x10a

goroutine 9725 [select, 25 minutes]:
runtime.gopark(0xc007e5fd98?, 0x2?, 0x80?, 0x56?, 0xc007e5fc8c?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc007e5fae8 sp=0xc007e5fac8 pc=0x43fd16
runtime.selectgo(0xc007e5fd98, 0xc007e5fc88, 0xc004dd2d40?, 0x0, 0x0?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc007e5fc28 sp=0xc007e5fae8 pc=0x4503be
github.com/couchbase/indexing/secondary/dcp.(*Client).runObserveStreamingEndpoint(0xc018f69128?, {0xc017dd43d8, 0x17}, 0xc018f69170, 0xc018f69128, 0xc0212fa960)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/dcp/pools.go:508 +0x73a fp=0xc007e5fee8 sp=0xc007e5fc28 pc=0x8645fa
github.com/couchbase/indexing/secondary/dcp.(*Client).RunObservePool(0xc00034c7e0?, {0x12616a0?, 0xc0028114c0?}, 0x16?, 0x1426750?)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/dcp/pools.go:396 +0xb2 fp=0xc007e5ff50 sp=0xc007e5fee8 pc=0x8636d2
github.com/couchbase/indexing/secondary/common.(*serviceNotifierInstance).RunPoolObserver(0xc00034c7e0)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/common/services_notifier.go:148 +0x56 fp=0xc007e5ffc8 sp=0xc007e5ff50 pc=0xc7adf6
github.com/couchbase/indexing/secondary/common.NewServicesChangeNotifier.func2()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/common/services_notifier.go:261 +0x26 fp=0xc007e5ffe0 sp=0xc007e5ffc8 pc=0xc7c026
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc007e5ffe8 sp=0xc007e5ffe0 pc=0x473161
created by github.com/couchbase/indexing/secondary/common.NewServicesChangeNotifier
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/common/services_notifier.go:261 +0x4d5

goroutine 5322 [select, 42 minutes]:
runtime.gopark(0xc004dd4f90?, 0x2?, 0xf8?, 0x4d?, 0xc004dd4f34?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc004dd4db0 sp=0xc004dd4d90 pc=0x43fd16
runtime.selectgo(0xc004dd4f90, 0xc004dd4f30, 0xc01a82bac0?, 0x0, 0xc01bbdc2a0?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc004dd4ef0 sp=0xc004dd4db0 pc=0x4503be
net/http.(*persistConn).writeLoop(0xc01a9ea900)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2410 +0xf2 fp=0xc004dd4fc8 sp=0xc004dd4ef0 pc=0x7a45b2
net/http.(*Transport).dialConn.func6()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x26 fp=0xc004dd4fe0 sp=0xc004dd4fc8 pc=0x7a1026
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc004dd4fe8 sp=0xc004dd4fe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x173d

goroutine 3157 [chan receive, 49 minutes]:
runtime.gopark(0xc002bce8a0?, 0x1e?, 0x1e?, 0x0?, 0xc00071bea8?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00071be28 sp=0xc00071be08 pc=0x43fd16
runtime.chanrecv(0xc0099b2b40, 0xc00071bfa0, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/chan.go:583 +0x49d fp=0xc00071beb8 sp=0xc00071be28 pc=0x40ca7d
runtime.chanrecv2(0xc0095f5900?, 0xc002bce8a0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/chan.go:447 +0x18 fp=0xc00071bee0 sp=0xc00071beb8 pc=0x40c5b8
github.com/couchbase/gometa/common.(*PeerPipe).doSend(0xc008459e00)
	/opt/build/goproj/src/github.com/couchbase/gometa/common/peerPipe.go:167 +0x9e fp=0xc00071bfc8 sp=0xc00071bee0 pc=0xd36cbe
github.com/couchbase/gometa/common.NewPeerPipe.func1()
	/opt/build/goproj/src/github.com/couchbase/gometa/common/peerPipe.go:65 +0x26 fp=0xc00071bfe0 sp=0xc00071bfc8 pc=0xd365a6
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00071bfe8 sp=0xc00071bfe0 pc=0x473161
created by github.com/couchbase/gometa/common.NewPeerPipe
	/opt/build/goproj/src/github.com/couchbase/gometa/common/peerPipe.go:65 +0x14d

goroutine 926 [GC worker (idle)]:
runtime.gopark(0x1e515a0?, 0x1?, 0xaf?, 0xef?, 0x413700?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000349f50 sp=0xc000349f30 pc=0x43fd16
runtime.gcBgMarkWorker()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc000349fe0 sp=0xc000349f50 pc=0x421091
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000349fe8 sp=0xc000349fe0 pc=0x473161
created by runtime.gcBgMarkStartWorkers
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1199 +0x25

goroutine 5422 [select, 42 minutes]:
runtime.gopark(0xc01e587f90?, 0x2?, 0xf8?, 0x7d?, 0xc01e587f34?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc01e587db0 sp=0xc01e587d90 pc=0x43fd16
runtime.selectgo(0xc01e587f90, 0xc01e587f30, 0xc01bc38c80?, 0x0, 0xc01c1844b0?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc01e587ef0 sp=0xc01e587db0 pc=0x4503be
net/http.(*persistConn).writeLoop(0xc01b9dd200)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2410 +0xf2 fp=0xc01e587fc8 sp=0xc01e587ef0 pc=0x7a45b2
net/http.(*Transport).dialConn.func6()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x26 fp=0xc01e587fe0 sp=0xc01e587fc8 pc=0x7a1026
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc01e587fe8 sp=0xc01e587fe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x173d

goroutine 5421 [select, 42 minutes]:
runtime.gopark(0xc01e586f68?, 0x4?, 0x3?, 0x0?, 0xc01e586db0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc01e586c00 sp=0xc01e586be0 pc=0x43fd16
runtime.selectgo(0xc01e586f68, 0xc01e586da8, 0xc01be28900?, 0x0, 0xcb73579207694801?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc01e586d40 sp=0xc01e586c00 pc=0x4503be
net/http.(*persistConn).readLoop(0xc01b9dd200)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2227 +0xd85 fp=0xc01e586fc8 sp=0xc01e586d40 pc=0x7a34c5
net/http.(*Transport).dialConn.func5()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x26 fp=0xc01e586fe0 sp=0xc01e586fc8 pc=0x7a1086
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc01e586fe8 sp=0xc01e586fe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x16ea

goroutine 1003 [select, 54 minutes]:
runtime.gopark(0xc00031bf68?, 0x4?, 0x3?, 0x0?, 0xc00031bdb0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00031bc00 sp=0xc00031bbe0 pc=0x43fd16
runtime.selectgo(0xc00031bf68, 0xc00031bda8, 0xc00ac00ec0?, 0x0, 0x1?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc00031bd40 sp=0xc00031bc00 pc=0x4503be
net/http.(*persistConn).readLoop(0xc007473440)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2227 +0xd85 fp=0xc00031bfc8 sp=0xc00031bd40 pc=0x7a34c5
net/http.(*Transport).dialConn.func5()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x26 fp=0xc00031bfe0 sp=0xc00031bfc8 pc=0x7a1086
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00031bfe8 sp=0xc00031bfe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x16ea

goroutine 5379 [select, 42 minutes]:
runtime.gopark(0xc01e475f68?, 0x4?, 0x3?, 0x0?, 0xc01e475db0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc01e475c00 sp=0xc01e475be0 pc=0x43fd16
runtime.selectgo(0xc01e475f68, 0xc01e475da8, 0xc0188e3d00?, 0x0, 0xc004abe501?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc01e475d40 sp=0xc01e475c00 pc=0x4503be
net/http.(*persistConn).readLoop(0xc01a9eac60)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2227 +0xd85 fp=0xc01e475fc8 sp=0xc01e475d40 pc=0x7a34c5
net/http.(*Transport).dialConn.func5()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x26 fp=0xc01e475fe0 sp=0xc01e475fc8 pc=0x7a1086
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc01e475fe8 sp=0xc01e475fe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x16ea

goroutine 5360 [select, 42 minutes]:
runtime.gopark(0xc01e4f8f90?, 0x2?, 0xf8?, 0x8d?, 0xc01e4f8f34?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc01e4f8db0 sp=0xc01e4f8d90 pc=0x43fd16
runtime.selectgo(0xc01e4f8f90, 0xc01e4f8f30, 0xc01bc38180?, 0x0, 0xc01bc14f00?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc01e4f8ef0 sp=0xc01e4f8db0 pc=0x4503be
net/http.(*persistConn).writeLoop(0xc01a9eaea0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2410 +0xf2 fp=0xc01e4f8fc8 sp=0xc01e4f8ef0 pc=0x7a45b2
net/http.(*Transport).dialConn.func6()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x26 fp=0xc01e4f8fe0 sp=0xc01e4f8fc8 pc=0x7a1026
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc01e4f8fe8 sp=0xc01e4f8fe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x173d

goroutine 1017 [GC worker (idle)]:
runtime.gopark(0xfe13455f997?, 0x1?, 0xae?, 0x55?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00a47b750 sp=0xc00a47b730 pc=0x43fd16
runtime.gcBgMarkWorker()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc00a47b7e0 sp=0xc00a47b750 pc=0x421091
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00a47b7e8 sp=0xc00a47b7e0 pc=0x473161
created by runtime.gcBgMarkStartWorkers
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1199 +0x25

goroutine 3173 [select]:
runtime.gopark(0xc00724df90?, 0x2?, 0xf0?, 0x75?, 0xc00724df4c?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00724ddc8 sp=0xc00724dda8 pc=0x43fd16
runtime.selectgo(0xc00724df90, 0xc00724df48, 0xc001059530?, 0x0, 0x100010100000000?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc00724df08 sp=0xc00724ddc8 pc=0x4503be
github.com/couchbase/gometa/protocol.(*Follower).startListener(0xc00764b8f0)
	/opt/build/goproj/src/github.com/couchbase/gometa/protocol/follower.go:149 +0xe5 fp=0xc00724dfc8 sp=0xc00724df08 pc=0xdee685
github.com/couchbase/gometa/protocol.(*Follower).Start.func1()
	/opt/build/goproj/src/github.com/couchbase/gometa/protocol/follower.go:76 +0x26 fp=0xc00724dfe0 sp=0xc00724dfc8 pc=0xdee286
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00724dfe8 sp=0xc00724dfe0 pc=0x473161
created by github.com/couchbase/gometa/protocol.(*Follower).Start
	/opt/build/goproj/src/github.com/couchbase/gometa/protocol/follower.go:76 +0x56

goroutine 1000 [select, 54 minutes]:
runtime.gopark(0xc0079ecf90?, 0x2?, 0xf8?, 0xcd?, 0xc0079ecf34?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc0079ecdb0 sp=0xc0079ecd90 pc=0x43fd16
runtime.selectgo(0xc0079ecf90, 0xc0079ecf30, 0xc00ac00d40?, 0x0, 0xc00ac16f90?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc0079ecef0 sp=0xc0079ecdb0 pc=0x4503be
net/http.(*persistConn).writeLoop(0xc007473320)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2410 +0xf2 fp=0xc0079ecfc8 sp=0xc0079ecef0 pc=0x7a45b2
net/http.(*Transport).dialConn.func6()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x26 fp=0xc0079ecfe0 sp=0xc0079ecfc8 pc=0x7a1026
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0079ecfe8 sp=0xc0079ecfe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x173d

goroutine 1015 [GC worker (idle)]:
runtime.gopark(0xfe12ab2838e?, 0x3?, 0xbc?, 0xa8?, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00999e750 sp=0xc00999e730 pc=0x43fd16
runtime.gcBgMarkWorker()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc00999e7e0 sp=0xc00999e750 pc=0x421091
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00999e7e8 sp=0xc00999e7e0 pc=0x473161
created by runtime.gcBgMarkStartWorkers
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1199 +0x25

goroutine 907 [select, 54 minutes]:
runtime.gopark(0xc00071ef90?, 0x2?, 0xf8?, 0xed?, 0xc00071ef34?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00071edb0 sp=0xc00071ed90 pc=0x43fd16
runtime.selectgo(0xc00071ef90, 0xc00071ef30, 0xc00ac00980?, 0x0, 0xc00ac16900?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc00071eef0 sp=0xc00071edb0 pc=0x4503be
net/http.(*persistConn).writeLoop(0xc0074726c0)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2410 +0xf2 fp=0xc00071efc8 sp=0xc00071eef0 pc=0x7a45b2
net/http.(*Transport).dialConn.func6()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x26 fp=0xc00071efe0 sp=0xc00071efc8 pc=0x7a1026
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00071efe8 sp=0xc00071efe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1766 +0x173d

goroutine 5417 [select, 42 minutes]:
runtime.gopark(0xc01e57af68?, 0x4?, 0x3?, 0x0?, 0xc01e57adb0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc01e57ac00 sp=0xc01e57abe0 pc=0x43fd16
runtime.selectgo(0xc01e57af68, 0xc01e57ada8, 0xc01be28280?, 0x0, 0xc029515401?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc01e57ad40 sp=0xc01e57ac00 pc=0x4503be
net/http.(*persistConn).readLoop(0xc01b9dcb40)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2227 +0xd85 fp=0xc01e57afc8 sp=0xc01e57ad40 pc=0x7a34c5
net/http.(*Transport).dialConn.func5()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x26 fp=0xc01e57afe0 sp=0xc01e57afc8 pc=0x7a1086
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc01e57afe8 sp=0xc01e57afe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x16ea

goroutine 858 [chan send, 54 minutes]:
runtime.gopark(0x0?, 0xc004ba1a98?, 0x67?, 0x3b?, 0x10?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc0054d9a68 sp=0xc0054d9a48 pc=0x43fd16
runtime.chansend(0xc009fde780, 0xc0054d9b78, 0x1, 0x0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/chan.go:259 +0x42e fp=0xc0054d9af0 sp=0xc0054d9a68 pc=0x40bc6e
runtime.chansend1(0x744360?, 0xc004ba199b?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/chan.go:145 +0x1d fp=0xc0054d9b20 sp=0xc0054d9af0 pc=0x40b81d
gopkg.in/couchbase/gocbcore%2ev7.(*Agent).connect.func2(0xc00a3a26c0?, {0x0?, 0x128fc1b?}, {0x141a300, 0xc00041e500})
	/opt/build/go/pkg/mod/gopkg.in/couchbase/gocbcore.v7@v7.1.18/agent.go:879 +0x11e fp=0xc0054d9b60 sp=0xc0054d9b20 pc=0xceec3e
gopkg.in/couchbase/gocbcore%2ev7.(*Agent).httpLooper.func1(0x1)
	/opt/build/go/pkg/mod/gopkg.in/couchbase/gocbcore.v7@v7.1.18/agenthttpcfg.go:136 +0x423 fp=0xc0054d9c58 sp=0xc0054d9b60 pc=0xcf3123
gopkg.in/couchbase/gocbcore%2ev7.(*Agent).httpLooper.func1(0x0)
	/opt/build/go/pkg/mod/gopkg.in/couchbase/gocbcore.v7@v7.1.18/agenthttpcfg.go:140 +0x44f fp=0xc0054d9d50 sp=0xc0054d9c58 pc=0xcf314f
gopkg.in/couchbase/gocbcore%2ev7.(*Agent).httpLooper(0xc00b8e1980, 0xc00a333f20)
	/opt/build/go/pkg/mod/gopkg.in/couchbase/gocbcore.v7@v7.1.18/agenthttpcfg.go:148 +0x35a fp=0xc0054d9fc0 sp=0xc0054d9d50 pc=0xcf21fa
gopkg.in/couchbase/gocbcore%2ev7.(*Agent).connect.func4()
	/opt/build/go/pkg/mod/gopkg.in/couchbase/gocbcore.v7@v7.1.18/agent.go:877 +0x2a fp=0xc0054d9fe0 sp=0xc0054d9fc0 pc=0xceeaea
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0054d9fe8 sp=0xc0054d9fe0 pc=0x473161
created by gopkg.in/couchbase/gocbcore%2ev7.(*Agent).connect
	/opt/build/go/pkg/mod/gopkg.in/couchbase/gocbcore.v7@v7.1.18/agent.go:877 +0xb1c

goroutine 9726 [select, 25 minutes]:
runtime.gopark(0xc00abd7d90?, 0x2?, 0x80?, 0x56?, 0xc00abd7c84?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc00abd7ae0 sp=0xc00abd7ac0 pc=0x43fd16
runtime.selectgo(0xc00abd7d90, 0xc00abd7c80, 0xc004dd2d38?, 0x0, 0x0?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc00abd7c20 sp=0xc00abd7ae0 pc=0x4503be
github.com/couchbase/indexing/secondary/dcp.(*Client).runObserveStreamingEndpoint(0xc0129c2400?, {0xc00a808bd0, 0x24}, 0xc018f691d0, 0xc018f691a0, 0xc0212fa960)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/dcp/pools.go:508 +0x73a fp=0xc00abd7ee0 sp=0xc00abd7c20 pc=0x8645fa
github.com/couchbase/indexing/secondary/dcp.(*Client).RunObserveNodeServices(0xc00034c7e0?, {0x12616a0?, 0xc0003e15e0?}, 0xc00ae39fd0?, 0xc00a998a20?)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/dcp/pools.go:412 +0xbc fp=0xc00abd7f50 sp=0xc00abd7ee0 pc=0x86399c
github.com/couchbase/indexing/secondary/common.(*serviceNotifierInstance).RunServicesObserver(0xc00034c7e0)
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/common/services_notifier.go:157 +0x54 fp=0xc00abd7fc8 sp=0xc00abd7f50 pc=0xc7af14
github.com/couchbase/indexing/secondary/common.NewServicesChangeNotifier.func3()
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/common/services_notifier.go:262 +0x26 fp=0xc00abd7fe0 sp=0xc00abd7fc8 pc=0xc7bfc6
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00abd7fe8 sp=0xc00abd7fe0 pc=0x473161
created by github.com/couchbase/indexing/secondary/common.NewServicesChangeNotifier
	/opt/build/goproj/src/github.com/couchbase/indexing/secondary/common/services_notifier.go:262 +0x51c

goroutine 5411 [select, 42 minutes]:
runtime.gopark(0xc01e578f68?, 0x4?, 0x3?, 0x0?, 0xc01e578db0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc01e578c00 sp=0xc01e578be0 pc=0x43fd16
runtime.selectgo(0xc01e578f68, 0xc01e578da8, 0xc01bd940c0?, 0x0, 0xc029515401?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc01e578d40 sp=0xc01e578c00 pc=0x4503be
net/http.(*persistConn).readLoop(0xc01b9dc900)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2227 +0xd85 fp=0xc01e578fc8 sp=0xc01e578d40 pc=0x7a34c5
net/http.(*Transport).dialConn.func5()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x26 fp=0xc01e578fe0 sp=0xc01e578fc8 pc=0x7a1086
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc01e578fe8 sp=0xc01e578fe0 pc=0x473161
created by net/http.(*Transport).dialConn
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x16ea

goroutine 923 [GC worker (idle)]:
runtime.gopark(0x1e515a0?, 0x1?, 0x2?, 0xed?, 0xc0003457d0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc000345750 sp=0xc000345730 pc=0x43fd16
runtime.gcBgMarkWorker()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1275 +0xf1 fp=0xc0003457e0 sp=0xc000345750 pc=0x421091
runtime.goexit()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc0003457e8 sp=0xc0003457e0 pc=0x473161
created by runtime.gcBgMarkStartWorkers
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/mgc.go:1199 +0x25

goroutine 995 [select, 54 minutes]:
runtime.gopark(0xc0068ddf68?, 0x4?, 0x3?, 0x0?, 0xc0068dddb0?)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/proc.go:381 +0xd6 fp=0xc0068ddc00 sp=0xc0068ddbe0 pc=0x43fd16
runtime.selectgo(0xc0068ddf68, 0xc0068ddda8, 0xc00ac00d80?, 0x0, 0x1?, 0x1)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/runtime/select.go:327 +0x7be fp=0xc0068ddd40 sp=0xc0068ddc00 pc=0x4503be
net/http.(*persistConn).readLoop(0xc007472d80)
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:2227 +0xd85 fp=0xc0068ddfc8 sp=0xc0068ddd40 pc=0x7a34c5
net/http.(*Transport).dialConn.func5()
	/home/buildbot/.cbdepscache/exploded/x86_64/go-1.20.6/go/src/net/http/transport.go:1765 +0x26 fp=0xc0068ddfe0 sp=0xc0068ddfc8 pc=0x7a1086
runtime.goexit()
	/home/buildbot/.cbdepscache/explod