OpenMLDB Versions Save

Features

• Improve the messages and errors when inserting rows (#2834 @vagetablechicken)
• Add a new configuration max_memory to limit the memory usage of a tablet (#2815 @dl239)
• Add new maintenance tools deploy-all and start-all (#2809 @zhanghaohit)
• Insertion returns errors if the value of timestamp field is negative (#2776 @dl239)
• Support the new built-in functions unix_timestamp, pmod, datediff, and size (#2843 #2839 #2847 #2864 @zhanghaohit)
• Add the new data type of ARRAY in UDFs (#2817 @aceforeverd)
• Improve the documents (#2868 @haseeb-xd, #2878 @Jake-00, #2879 #2876 #2889 #2890 @vagetablechicken, #2881 #2907 #2908 #2897 @selenachenjingxin, #2859 @AdvancedUno, #2893 @lumianph, #2916 @aceforeverd)

Bug Fixes

• Window over a subquery(t1 LAST JOIN t2) fails due to column renaming in the subquery. (#2739 @aceforeverd)
• SHOW JOBLOG fails under certain circumstances. (#2874 @tobegit3hub)
• OP is not deleted if the related table has been dropped. (#2548 @dl239)
• Memory is not released when deleting an index in some cases. (#2806 @dl239)
• Changing a leader to a specified endpoint fails if there are data writing. (#2858 @dl239)
• UDFs do not work for yarn-client and yarn-cluster in the offline mode. (#2802 @tobegit3hub)
• Other minor bug fixes (#2828 #2903 #2906 @vagetablechicken, #2867 #2912 @dl239)

Code Refactoring

#2860 @mammar11, #2865 #2863 @vagetablechicken, #2861 #2862 #2871 @Ziy1-Tan

Features

• Add pre-upgrade and post-upgrade options in tools for upgrade (#2761 @zhanghaohit)
• Support starting OpenMLDB with daemon (#2833 @vagetablechicken)
• Improve the documents (#2827 @tobegit3hub, #2838 @vagetablechicken)

Bug Fixes

• The CLI may crash if executing show job without a job ID. (#2771 @aceforeverd)
• select count(*) may return empty result after tablets restart. (#2835 @zhanghaohit)
• A tablet may crash if the output of SQL engine runner is null. (#2831 @dl239)

Features

• Support the where clause in the SQL batch engine (#2820 @tobegit3hub)
• Support input and output with the JSON format in APIServer (#2813 @vagetablechicken)
• Improve the documents (#2814 @vagetablechicken)

Code Refactoring

#2816 @dl239, #2714 @aceforeverd

Features

• Support importing and exporting data from/to Hive (@2778 @vagetablechicken)
• Improve the module of autofe (#2777 @vagetablechicken)
• Improve error messages of the TaskManager client (#2780 @vagetablechicken)
• Improve the documents (#2781 @zhanghaohit, #2767 #2792 @vagetablechicken, #2805 @selenachenjingxin, #2810 @dl239)

Bug Fixes

• Python SDK workflow may fail on MacOS. (#2783 @vagetablechicken, #2788 @dl239)
• There are syntax errors in some log messages. (@2770 dl239)
• Installing Python SDK requires unnessary packages. (#2791 @vagetablechicken)

Features

• Support the new build-in function hash64 (#2754 @aceforeverd)
• Improve the documents (#2763 @dl239, #2610 #2606 @michelle-qinqin)

Bug Fixes

• pytest command is not found in the MacOS virtual machine. (#2765 @tobegit3hub)
• Wrong output schema passes to the WindowAggRunner. (#2758 @aceforeverd)
• There are no outputs when executing showopstatus command if no database is specified (#2773 @dl239)
• The data recovery tool fails in some cases (#2768 @dl239)

Features

• Optimize the distribution of table partitions (#2701 @jkpjkpjkp)
• Add a new workflow to generate the documents of built-in functions automatically (#2709 #2729 @aceforeverd)
• Support the new SQL statement show joblog (#2732 @aceforeverd, #2747 @tobegit3hub)
• Add a warning message for show table status (#2738 @zhanghaohit)
• Add a new tool for data recovery and scale-out/scale-in (#2736 @dl239)
• Improve the documents (#2707 #2727 @aceforeverd, #2718 #2538 #2731 #2752 @vagetablechicken, #2607 #2609 @michelle-qinqin, #2733 @zhanghaohit, #2742 @auula)

Bug Fixes

• Incorrect data will be loaded in offline mode if the schema mismatches with parquet files. (#2648 @vagetablechicken)
• Creating index fails if specifying a database in SQL statement (#2720 @dl239)
• start_time is not human-readable after submitting a job (#2751 @tobegit3hub)
• Incorrect result of GetRecordIdxCnt is produced in MemTable (#2719 @jkpjkpjkp)

Code Refactoring

#2688 #2717 @vagetablechicken, #2705 #2728 @dl239, #2601 @team-317, #2737 @Jake-00

Features

• Support a new series of built-in functions top_n_value_*_cate_where (#2622 @aceforeverd)
• Support online batch computation and aggregation over a full table (#2620 @zhanghaohit)
• Support load_mode and thread options for LOAD DATA (#2684 @zhanghaohit)
• Improve the documents (#2476, #2486 #2514 #2611 #2693 #2462 @michelle-qinqin, #2695 @lumianph, #2653 @vagetablechicken)
• Support running MacOS compiling jobs in the CICD workflow (#2665 @dl239)

Bug Fixes

• Recreating index fails if it has been dropped. (#2440 @dl239)
• The traverse method may get duplicate data if there are same ts records on one pk (#2637 @dl239)
• Multiple window union will fail when compiling in batch mode (#2478 @tobegit3hub, #2561 @aceforeverd)
• select * ... statement may cause inconsistent output schemas in many cases (#2660 @aceforeverd)
• Result is incorrect if the window is specified as UNBOUNDED PRECEDING AND CURRENT ROW EXCLUDE CURRENT_ROW (#2674 @aceforeverd)
• Incorrect slice offsets may lead to offline jobs hang (#2687 @aceforeverd)
• Other minor bug fixes (#2669 @dl239, #2683 @zhanghaohit)

Code Refactoring

#2541 @dl239, #2573 #2672 @haseeb-xd

Features

• Support setting the configuration of glog for clients (#2482 @vagetablechicken)
• Add the checksum of SHA256 for release packages (#2560 @team-317)
• Support the new build-in function unhex (#2431 @aucker)
• Support the readable date and time format in CLI (#2568 @dl239)
• Support the LAST JOIN with a subquery as a producer of window node in the request mode (#2569 @aceforeverd)
• Upgrade the Spark version to 3.2.1 (#2566 @tobegit3hub, #2635 @dl239)
• Support setting the SQL cache size in SDKs (#2605 @vagetablechicken)
• Add a new interface of ValidateSQL to validate the syntax of SQL (#2626 @vagetablechicken)
• Improve the documents (#2405 #2492 #2562 #2496 #2495 #2436 #2487 #2623 @michelle-qinqin, #2543 @linjing-lab, #2584 @JourneyGo, #2567 #2583 @vagetablechicken, #2643 @dl239)
• Other minor features (#2504 #2572 #2498 #2598 @aceforeverd, #2555 #2641 @tobegit3hub, #2550 @zhanghaohit, #2595 @Elliezza, #2592 @vagetablechicken)

Bug Fixes

• After a nameserver restarting, deployments may not recover. (#2533 @dl239)
• If the type of first column is bool, it fails to resolve the function count_where. (#2570 @aceforeverd)
• Other minor bug fixes (#2540 #2577 #2625 #2655 @dl239, #2585 @snehalsenapati23, #2539 @vagetablechicken)

Code Refactoring

#2516 #2520 #2522 #2521 #2542 #2531 #2581 @haseeb-xd, #2525 #2526 #2527 #2528 @kstrifonoff, #2523 @ighmaZ, #2546 #2549 @NevilleMthw, #2559 @marandabui, #2554 @gokullan, #2580 @team-317, #2599 @lbartyczak, #2594 @shivamgupta-sg, #2571 @Jake-00

Features

• Support independently executing the OpenMLDB offline engine without the OpenMLDB deployment (#2423 @tobegit3hub)
• Support the log setting of ZooKeeper and disable ZooKeeper logs in the diagnostic tool (#2451 @vagetablechicken)
• Support query parameters of the SQL query APIs (#2277 @qsliu2017)
• Improve the documents (#2406 @aceforeverd, #2408 #2414 @vagetablechicken, #2410 #2402 #2356 #2374 #2396 #2376 #2419 @michelle-qinqin, #2424 #2418 @dl239, #2455 @lumianph, #2458 @tobegit3hub)
• Other minor features (#2420 @aceforeverd, #2411 @wuyou10206, #2446 #2452 @vagetablechicken, #2475 @tobegit3hub)

Bug Fixes

• Table creation succeeds even if partitionnum is set to 0, which should report an error. (#2220 @dl239)
• There are thread races in aggregators if there are concurrent puts. (#2472 @zhanghaohit)
• The limit clause dose not work if it is used with the where and group by clauses. (#2447 @aceforeverd)
• The TaskManager process will terminate if ZooKeeper disconnects. (#2494 @tobegit3hub)
• The replica cluster dose not create the database if a database is created in the leader cluster (#2488 @dl239)
• When there is data in base tables, deployment with long windows still can be executed (which should report an error). (#2501 @zhanghaohit)
• Other minor bug fixes (#2415 @aceforeverd, #2417 #2434 #2435 #2473 @dl239, #2466 @vagetablechicken)

Code Refactoring

#2413 @dl239, #2470 #2467 #2468 @vagetablechicken

Features

• Support new build-in functions last_day and regexp_like (#2262 @HeZean, #2187 @jiang1997)
• Support Jupyter Notebook for the TalkingData use case (#2354 @vagetablechicken)
• Add a new API to disable Saprk logs of the batch engine (#2359 @tobegit3hub)
• Add the use case of precision marketing based on OneFlow (#2267 @Elliezza @vagetablechicken @siqi)
• Support the RPC request timeout in CLI and Python SDK (#2371 @vagetablechicken)
• Improve the documents (#2021 @liuceyi, #2348 #2316 #2324 #2361 #2315 #2323 #2355 #2328 #2360 #2378 #2319 #2350 #2395 #2398 @michelle-qinqin, #2373 @njzyfr, #2370 @tobegit3hub, #2367 #2382 #2375 #2401 @vagetablechicken, #2387 #2394 @dl239, #2379 @aceforeverd, #2403 @lumianph, #2400 @gitpod-for-oss @aceforeverd, )
• Other minor features (#2363 @aceforeverd, #2185 @qsliu2017)

Bug Fixes

• APIServer will core dump if no rs in QueryResp. (#2346 @vagetablechicken)
• Data has not been deleted from pre-aggr tables if there are delete operations in a main table. (#2300 @zhanghaohit)
• Task jobs will core dump when enabling UnsafeRowOpt with multiple threads in the Yarn cluster. (#2352 #2364 @tobegit3hub)
• Other minor bug fixes (#2336 @dl239, #2337 @dl239, #2385 #2372 @aceforeverd, #2383 #2384 @vagetablechicken)

Code Refactoring

#2310 @hv789, #2306 #2305 @yeya24, #2311 @Mattt47, #2368 @TBCCC, #2391 @PrajwalBorkar, #2392 @zahyaah, #2405 @wang-jiahua