Image for post
Image for post

Industry: Banking

Author: Yu Han (Engineer at Information Technology Operations Center, Bank of China)

Bank of China is the fourth largest state-owned commercial bank in China. Since 2016, we’ve been using Zabbix, a popular open-source monitoring solution, to monitor our IT infrastructure. We used to use MySQL as the backend storage for Zabbix; however, MySQL is not scalable enough to monitor IT environments on a large scale. After trying different solutions, we chose TiDB, a MySQL-compatible, open-source, distributed SQL database to replace MySQL as the backend for a large-scale Zabbix. (We call this collaboration “TiZabbix”.) …


Jun Yu

Image for post
Image for post

For financial companies, especially in the banking, securities, and insurance industries, a major IT concern is how to prevent or respond to failures: hardware failures, network failures — any sudden failure that may affect the business. A core component of these businesses is the back-end database. It must operate safely and ensure data security without exceptions. To achieve this level of business continuity, the database service must be highly available. However, high availability alone is not enough. We must consider the business impact caused by disasters in the server room, the data center, or even the whole city. …


Shuang Chen

Image for post
Image for post

TiDB, an open-source, distributed SQL database, provides detailed monitoring metrics through Prometheus and Grafana. These metrics are often the key to troubleshooting performance problems in the cluster.

However, for novice TiDB users, understanding hundreds of monitoring metrics can be overwhelming. You may wonder:

TiDB 4.0.7 introduces a new feature in its web UI TiDB Dashboard: the metrics relation graph. It provides a tree diagram of the TiDB cluster performance metrics, enabling users to quickly see the relationships between TiDB internal processes and to get a new perspective on the cluster status. …


Author: Zhi Qi (Real-time Analytics R&D engineer at PingCAP)

Image for post
Image for post

When a data-driven company grows to a certain size, traditional data storage can no longer meet its needs. For real-time business intelligence, you need a real-time data warehouse. Companies can use real-time data warehouses to implement real-time Online Analytical Processing (OLAP) analytics, real-time data panels, real-time application monitoring, and real-time data interface services.

Some people think that a real-time data warehouse architecture is complex and difficult to operate and maintain. Today, I will explain why that isn’t true. Apache Flink is a framework and distributed processing engine for stateful computations over unbounded and bounded data streams. TiDB is an open-source, distributed, Hybrid Transactional/Analytical Processing (HTAP) database. Thanks to Flink 1.11’s …


Image for post
Image for post

With great pleasure, we are announcing that we have raised $270 million in Series D funding. This round is jointly led by GGV Capital, Access Technology Ventures, Anatole Investment, Jeneration Capital, and 5Y Capital (formerly Morningside Venture Capital). Coatue, Bertelsmann Asia Investment Fund (BAI), FutureX Capital, Kunlun Capital, Trustbridge Partners, and existing investors Matrix Partners China and Yunqi Partners also participated in this round. We are thrilled and humbled to have these prestigious and visionary investors join us as we strive to build the database of the future.

PingCAP began five years ago when our cofounders, sick and tired of the way databases were managed, scaled, and maintained, decided to build one themselves. Since day one, our mission has been to help every business unleash the power of data. With the help of a first-class team and hundreds of contributors from around the globe, we live our mission by delivering TiDB, an open-source distributed NewSQL Hybrid Transactional/Analytical Processing (HTAP) database. Over the last five years, more than 1,500 companies around the world have adopted TiDB, including Square, PayPay, BookMyShow, Dailymotion, Bank of China, JD.com, and Shopee. These companies and many others use TiDB for tasks like solving the scalability bottlenecks for the online transactional businesses, integrating TiDB as the real-time analytics platform, or saving infrastructure and maintenance cost (by migrating to TiDB Cloud). …


Image for post
Image for post

Our digital society produces vast amounts of data, and traditional databases such as MySQL face performance and capacity bottlenecks. One solution, database sharding, is complex and brings high operation and maintenance costs.

TiDB is an open-source, distributed, Hybrid Transactional/Analytical Processing (HTAP) database. It is an enterprise-level NewSQL database that separates the computing and storage layers. TiDB can elastically scale based on the needs of your applications, making them more real-time and intelligent. TiDB provides TiDB Data Migration (DM) to help users migrate data from MySQL to TiDB and effectively reduce migration costs and risks.

DM is an integrated data migration task management platform that supports full data migration and incremental data replication from MySQL or MariaDB to TiDB. DM 2.0 has reached general availability (GA), and it supports new enterprise-level…


Image for post
Image for post

Industry: Media and Entertainment

Author: Birong Huang (Senior engineer at U-Next)

U-Next is a subscription-based video streaming platform that has one of the largest market shares in Japan. In the past few years, our business grew rapidly, and our old IT infrastructure could no longer keep pace. We needed to upgrade our system.

Our previous MySQL cluster was hard to scale, and, when the server experienced high concurrency, the cluster had high latency. To solve these problems, we migrated our data to TiDB, an open-source, distributed SQL database that provides horizontal scalability and high performance. It speaks the MySQL protocol and works perfectly on our ARM architecture. …


Xianlin Chen (DBA at PalFish)

Image for post
Image for post

Recently, VLDB 2020 published PingCAP’s paper, TiDB: A Raft-based HTAP Database. This is the first paper in the industry to describe the implementation of a distributed Hybrid Transactional/Analytical Processing (HTAP) database. As a DBA who benefits greatly from TiDB, an open-source, distributed SQL database, I’m happy that VLDB recognized TiDB, and I’m inspired by the PingCAP engineering team’s novel ideas.

PingCAP’s paper is not the typical theoretical research paper, proposing an idea that may never be implemented. Instead, it proves, clearly and pragmatically, that a distributed HTAP database is achievable. …


Image for post
Image for post

Industry: Logistics

Author: Youzhi Zhu (Big Data Architect at ZTO Express)

ZTO Express is a leading express delivery company in China and one of the largest express delivery companies in the world. As of December 31, 2016, we provide express delivery service as well as other value-added logistics services in China, covering over 96% of China’s cities and counties. On Nov 11, 2019, the day of an annual Chinese shopping festival, our sales promotion achieved over 200 million orders.

As our business quickly grew, huge amounts of data flooded into our database. Oracle Exadata didn’t meet our requirement for data storage. After we sharded our database, we couldn’t perform data analytics in real time, and our database couldn’t scale. Two other options, Apache Kudu and HBase were not desirable for building a real-time data warehouse. We looked for a database that supported horizontal scalability, distributed transactions with strong consistency, highly concurrent writes, and minute-level, multi-dimensional queries. …


Image for post
Image for post

TiDB 4.0 introduces TiCDC as TiDB’s change data capture framework. It’s an open-source feature that replicates TiDB’s incremental changes to downstream platforms by subscribing to change logs in TiKV (TiDB’s storage engine). It can restore data to a consistent state with any upstream timestamp Oracle (TSO) and provides the TiCDC Open Protocol to support other data consumers that subscribe to TiKV’s data changes.

With high data reliability and horizontal scalability features, TiCDC provides high-availability replication services for 100 TB clusters with only milliseconds of latency. In TiDB 4.0.6, …

About

PingCAP

PingCAP is the team behind TiDB, an open source MySQL compatible NewSQL HTAP database. Official website: https://pingcap.com/ GitHub: https://github.com/pingcap

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store