Industry: Automobile

Authors: Kevin Ge, Jiansheng Han, and Sheyang Wang (TiDB virtual team at Chehaoduo)

Chehaoduo is an online trading platform for both new cars and personal used cars. Founded in 2015, it is now one of the largest auto trading platforms in China, valued at $9 billion in its series D round of funding last year.

In the early stages of Chehaoduo, to quickly adapt to our application development, we chose MySQL as our major database. However, as our business evolved, we were greatly troubled by the complication of MySQL sharding and schema changes. …

Today, we are excited to introduce the TiDB Incubator Program, a program designed to ensure that new projects in the TiDB ecosystem can obtain resources and help from the community towards their desired maturity level.

About the TiDB Incubator Program

The TiDB community is composed of developers, users, and partners in the TiDB ecosystem, and we’re dedicated to building “the database of the future” and the sustainable ecosystem for it. With the continued efforts, love, and support from the entire community, the TiDB ecosystem continues to evolve. …

Keao Yang — Maintainer of Chaos Mesh

In a production environment, filesystem faults might occur due to various incidents such as disk failures and administrator errors. As a Chaos Engineering platform, Chaos Mesh has supported simulating I/O faults in a filesystem ever since its early versions. By simply adding an IOChaos CustomResourceDefinition (CRD), we can watch how the filesystem fails and returns errors.

However, before Chaos Mesh 1.0, this experiment was not easy and may have consumed a lot of resources. We needed to inject sidecar containers to the Pod through the mutating admission webhooks and rewrite the ENTRYPOINT command. …

Coco Yi

I use open source projects on GitHub every day, and I’ve often wondered: Where do open source communities live?

Previously for different projects, I might have different answers in mind:

• GitHub: because that’s where the source code is and I can contribute to the project by opening issues and pull requests.
• Slack/Gitter/Google Groups/Self-hosted forums: because that’s where I chat and discuss with other contributors and feel belonged to a community.
• Stack Overflow: because sometimes it’s the only place that I can get quick support for my usage questions.

It wasn’t until GitHub recently introduced the Discussions feature that I found my only…

Industry: Banking

Authors:

• Panpan Hu (Database Platform Manager at WeBank)
• Wei Huang (Senior DBA of Database Platform at WeBank)

WeBank is China’s first privately-owned internet bank. It is backed by reputable companies such as Tencent. It offers convenient and high-quality financial services to underbanked individuals as well as small- and medium-sized enterprises. So far, we’ve served 250 million+ individual customers, 20 million+ individual business customers, and 1.5 million+ corporate customers.

As our businesses grew, our data size soared, and we encountered database performance and capacity bottlenecks. After we compared multiple solutions, we chose TiDB, an open-source, MySQL-compatible, NewSQL database. Thanks to its horizontal scalability, in a financing system, our batch processing time has been reduced by about 58%. …

Wenbo Zhang

If you need to dynamically trace Linux process system calls, you might first consider strace. strace is simple to use and works well for issues such as “Why can’t the software run on this machine?” However, if you’re running a trace in a production environment, strace is NOT a good choice. It introduces a substantial amount of overhead. According to a performance test conducted by Arnaldo Carvalho de Melo, a senior software engineer at Red Hat, the process traced using strace ran 173 times slower, which is disastrous for a production environment.

So are there any tools that excel at tracing system calls in a production environment? The answer is YES. This blog post introduces perf and traceloop, two commonly used command-line tools, to help you trace system calls in a production environment. …

Wenbo Zhang

At the beginning of 2020, when I used the BCC tools to analyze our database performance bottlenecks, and pulled the code from the GitHub, I accidentally discovered that there was an additional libbpf-tools directory in the BCC project. I had read an article on BPF portability and another on BCC to libbpf conversion, and I used what I learned to convert my previously submitted bcc-tools to libbpf-tools. I ended up converting nearly 20 tools. (See Why We Switched from bcc-tools to libbpf-tools for BPF Performance Analysis.)

During this process, I was fortunate to get a lot of help from Andrii Nakryiko (the libbpf + BPF CO-RE project’s leader). It was fun and I learned a lot. In this post, I’ll share my experience about writing Berkeley Packet Filter (BPF) applications with libbpf. I hope this article is helpful to people who are interested in libbpf and inspires them to further develop and improve BPF applications with libbpf. …

Wenbo Zhang

Linux’s memory management system is transparent to the user. However, if you’re not familiar with its working principles, you might meet unexpected performance issues. That’s especially true for sophisticated software like databases. When databases are running in Linux, even small system variations might impact performance.

After an in-depth investigation, we found that Transparent Huge Page (THP), a Linux memory management feature, often slows down database performance. In this post, I’ll describe how THP causes performance to fluctuate, the typical symptoms, and our recommended solutions.

What is THP

THP is an important feature of the Linux kernel. It maps page table entries to larger page sizes to reduce page faults. This improves the translation lookaside buffer (TLB) hit ratio. TLB is a memory cache used by the memory management unit to improve the translation speed from virtual memory addresses to physical memory addresses. …