58.com is China’s leading online marketplace for classifieds covering various categories, such as jobs, real estate, automotive, financing, used goods, and local services. Merchants and consumers can publish their advertisements on our online platform so that they can connect, share information, and conduct business. By the end of 2018, we had more than 500 million users, and our total revenue in 2019 was nearly US $2.23 billion.
As our businesses grew, large amounts of data flooded into our databases. But standalone MySQL databases couldn’t store so much data, and sharding was an undesirable solution. To achieve MySQL high availability, we needed an external tool. To perform Online Analytical Processing (OLAP), we had to use complicated and tedious extract, transform, load (ETL) jobs. …
Authors: Kevin Ge, Jiansheng Han, and Sheyang Wang (TiDB virtual team at Chehaoduo)
Chehaoduo is an online trading platform for both new cars and personal used cars. Founded in 2015, it is now one of the largest auto trading platforms in China, valued at $9 billion in its series D round of funding last year.
In the early stages of Chehaoduo, to quickly adapt to our application development, we chose MySQL as our major database. However, as our business evolved, we were greatly troubled by the complication of MySQL sharding and schema changes. …
Today, we are excited to introduce the TiDB Incubator Program, a program designed to ensure that new projects in the TiDB ecosystem can obtain resources and help from the community towards their desired maturity level.
The TiDB community is composed of developers, users, and partners in the TiDB ecosystem, and we’re dedicated to building “the database of the future” and the sustainable ecosystem for it. With the continued efforts, love, and support from the entire community, the TiDB ecosystem continues to evolve. …
Keao Yang — Maintainer of Chaos Mesh
In a production environment, filesystem faults might occur due to various incidents such as disk failures and administrator errors. As a Chaos Engineering platform, Chaos Mesh has supported simulating I/O faults in a filesystem ever since its early versions. By simply adding an IOChaos CustomResourceDefinition (CRD), we can watch how the filesystem fails and returns errors.
However, before Chaos Mesh 1.0, this experiment was not easy and may have consumed a lot of resources. We needed to inject sidecar containers to the Pod through the mutating admission webhooks and rewrite the
ENTRYPOINT command. …
I use open source projects on GitHub every day, and I’ve often wondered: Where do open source communities live?
Previously for different projects, I might have different answers in mind:
It wasn’t until GitHub recently introduced the Discussions feature that I found my only…
WeBank is China’s first privately-owned internet bank. It is backed by reputable companies such as Tencent. It offers convenient and high-quality financial services to underbanked individuals as well as small- and medium-sized enterprises. So far, we’ve served 250 million+ individual customers, 20 million+ individual business customers, and 1.5 million+ corporate customers.
As our businesses grew, our data size soared, and we encountered database performance and capacity bottlenecks. After we compared multiple solutions, we chose TiDB, an open-source, MySQL-compatible, NewSQL database. Thanks to its horizontal scalability, in a financing system, our batch processing time has been reduced by about 58%. …
If you need to dynamically trace Linux process system calls, you might first consider strace. strace is simple to use and works well for issues such as “Why can’t the software run on this machine?” However, if you’re running a trace in a production environment, strace is NOT a good choice. It introduces a substantial amount of overhead. According to a performance test conducted by Arnaldo Carvalho de Melo, a senior software engineer at Red Hat, the process traced using strace ran 173 times slower, which is disastrous for a production environment.
So are there any tools that excel at tracing system calls in a production environment? The answer is YES. This blog post introduces perf and traceloop, two commonly used command-line tools, to help you trace system calls in a production environment. …
At the beginning of 2020, when I used the BCC tools to analyze our database performance bottlenecks, and pulled the code from the GitHub, I accidentally discovered that there was an additional libbpf-tools directory in the BCC project. I had read an article on BPF portability and another on BCC to libbpf conversion, and I used what I learned to convert my previously submitted bcc-tools to libbpf-tools. I ended up converting nearly 20 tools. (See Why We Switched from bcc-tools to libbpf-tools for BPF Performance Analysis.)
During this process, I was fortunate to get a lot of help from Andrii Nakryiko (the libbpf + BPF CO-RE project’s leader). It was fun and I learned a lot. In this post, I’ll share my experience about writing Berkeley Packet Filter (BPF) applications with libbpf. I hope this article is helpful to people who are interested in libbpf and inspires them to further develop and improve BPF applications with libbpf. …
Linux’s memory management system is transparent to the user. However, if you’re not familiar with its working principles, you might meet unexpected performance issues. That’s especially true for sophisticated software like databases. When databases are running in Linux, even small system variations might impact performance.
After an in-depth investigation, we found that Transparent Huge Page (THP), a Linux memory management feature, often slows down database performance. In this post, I’ll describe how THP causes performance to fluctuate, the typical symptoms, and our recommended solutions.
THP is an important feature of the Linux kernel. It maps page table entries to larger page sizes to reduce page faults. This improves the translation lookaside buffer (TLB) hit ratio. TLB is a memory cache used by the memory management unit to improve the translation speed from virtual memory addresses to physical memory addresses. …
2020 has been a difficult year for everyone, but at PingCAP we continued to innovate and inform. This year, we released TiDB 4.0, a real-time Hybrid Transactional/Analytical Processing (HTAP) database, and we published 50+ technical blog posts and 17 case studies. We’d like to thank all our TiDB contributors and users for their trust and support. They not only helped us improve our product, but they also contributed great material for many of our excellent articles.
From pondering the future of databases, to intrepid bug hunters, to TiDB case studies, let’s take a look back at our 10 most popular posts in…