First, let’s clarify what kind of distributed systems we’re discussing here. Simply put, they need to satisfy two conditions: multi-node and stateful. Multi-node is straightforward to understand. Being stateful means the system needs to maintain some data — otherwise, we could simply scale horizontally without limit, and there would be no distributed systems problem to speak of.

Common distributed systems — whether databases like MySQL, Cassandra, HBase, message queues like RocketMQ, Kafka, Pulsar, or infrastructure like ZooKeeper — all satisfy these two conditions.

The implementation of these distributed systems typically needs to address two aspects: first, the system’s own functionality; and second, maintaining good performance and stability in a distributed environment. Even two systems with completely different functions will have many similarities in how they address the second type of problem. This article focuses on the second type.

Core Challenges of Distributed Systems

Next, let’s enumerate the common goals of distributed systems, including but not limited to:

• A large number of ordinary servers interconnected via a network, collectively providing services to the outside world.
• As the cluster scales up, overall system performance scales linearly.
• Automatic fault tolerance — failed nodes are automatically migrated, and data consistency across different nodes is maintained.

What are the challenges in achieving these goals? Here are the main ones:

1. Process crashes: Caused by many factors, including hardware failures, software bugs, routine maintenance, and in cloud environments, more complex reasons. The biggest problem with process crashes is data loss. For performance reasons, we often don’t write synchronously to disk — instead, we temporarily buffer data in memory and periodically flush to disk. When a process crashes, the data in memory buffers is clearly lost.

2. Network delays and interruptions: When communication between nodes becomes very slow, how does one node determine whether another node is healthy?

3. Network partitions: The cluster’s nodes split into two subsets — communication within each subset works, but communication between subsets is broken (brain split). How should the cluster continue to serve requests?

   A brief digression: under the CAP theorem, real-world systems typically adopt one of two modes — CP mode (giving up high availability) or AP mode (giving up strong consistency). Why is there no CA mode that gives up partition tolerance? Because we cannot assume network communication will always be reliable. Once a cluster splits into two partitions, re-merging is not realistic.

4. Process pauses: For example, a full GC may cause a process to become temporarily unavailable before quickly recovering. During the unavailable period, the cluster may have already reacted accordingly. How should state consistency be maintained when the node comes back?

5. Clock skew and message reordering: We want operations across different nodes in the cluster to have a clear ordering. Clock skew between nodes prevents us from using timestamps to guarantee this. Message reordering further complicates distributed system processing.

Now, let’s walk through the solutions to these problems.

Partitioning and Replication

For process crashes, the key point is that simply ensuring no data loss during a crash is not difficult at all — the real challenge is achieving this while maintaining system performance.

The first pattern to introduce is Write-Ahead Log (WAL): the server stores every state change as a command in an append-only file on disk. Since append operations are sequential disk writes, they are typically very fast and can be completed without impacting performance. During server failure recovery, the log can be replayed to rebuild in-memory state.

The core idea is to first persist data in a low-cost way (not necessarily limited to sequential disk writes), then acknowledge the client without blocking other operations. The server asynchronously performs the subsequent high-cost work.

Typical scenarios and variants: MySQL redo log; Redis AOF; Kafka itself; common practices in application development — for time-consuming operations, first write a database record indicating the task will be executed, then perform the actual task asynchronously.

WAL introduces a minor issue: the log keeps growing. How to handle its own storage? Two natural approaches: segmentation and cleanup.

Segmentation splits a large log into smaller ones. Since WAL logic is generally simple, splitting is not complex — much easier than typical database sharding. This pattern is called Segmented Log, and the canonical implementation is Kafka’s partitions.

For cleanup, there is a pattern called low-water mark: a marker indicating which parts of the log can be cleaned up. The marking can be based on data status (redo log), preset retention time (Kafka), or more fine-grained cleanup and compaction (AOF).

Consistency Protocols

Moving to network-related issues, a simple heartbeat pattern solves inter-node state synchronization. If no heartbeat is received within a time window, the node is considered down.

For brain-split scenarios, the Quorum pattern is typically used. It requires the number of live nodes in the cluster to reach a Quorum value (typically, with 2f+1 nodes, at most f nodes can go down, so the Quorum value is f+1) before the cluster can serve requests. When examining implementations of distributed systems like RocketMQ or ZooKeeper, you’ll find the requirement for a minimum number of live nodes — this is exactly the Quorum pattern.

Quorum solves the data durability problem — successfully written data will not be lost even if nodes fail. However, Quorum alone cannot guarantee strong consistency, because data on different nodes may differ in time. When clients connect to different nodes, they may see different results. This can be solved through the Leader and Followers pattern, where one node is elected as the leader to coordinate data replication across nodes and determine which data is visible to clients.

The High-Water Mark pattern determines which data is visible to clients. Generally, after a write is completed on a Quorum number of follower nodes, the data can be marked as visible to clients. The line of completed replication is the high-water mark.

The Leader-Followers pattern is so widely used that examples are unnecessary. There are many distributed election algorithms — Bully, ZAB, Paxos, Raft, among others. Paxos is difficult both to understand and implement. Bully triggers frequent elections when nodes go up and down frequently. Raft, on the other hand, strikes a relatively balanced trade-off in terms of stability, implementation complexity, and adoption — it is the most widely used distributed election algorithm. For example, Elasticsearch replaced Bully with Raft in version 7.0; Kafka 2.8 switched from ZooKeeper’s ZAB protocol to Raft.

At this point, let’s summarize. Essentially, an operation on a distributed system can be概括为 the following steps:

1. Write to the leader’s Write-Ahead Log
2. Write to 1 follower’s WAL
3. Write to the leader’s data store
4. Write to 1 follower’s data store
5. Write to a Quorum of followers’ WALs
6. Write to a Quorum of followers’ data stores

The order of steps 2–5 is not fixed. The most important way a distributed system balances performance and stability is, in essence, deciding the order of these operations and when to return a success acknowledgment to the client. For example, MySQL’s synchronous replication, asynchronous replication, and semi-synchronous replication are classic examples of these different orderings.

Fault Detection and Recovery

For process pauses, the main problem scenario is: suppose the leader pauses. During the pause, a new leader is elected. When the original leader recovers, what should happen? The Generation Clock pattern addresses this — simply put, each leader is assigned a monotonically increasing generation number to indicate which “generation” of leader it is. Concepts like Raft’s “term” and ZAB’s “epoch” are implementations of this Generation Clock idea.

For clock skew, in a distributed environment, clock differences between nodes are inevitable. Under the Leader-Followers pattern, this problem is already largely minimized. Many systems choose to perform all operations on the leader, with leader-follower replication taking the form of replicating and replaying logs. In this way, clock issues generally don’t need to be considered. The only potential issue arises during leader failover, where the old leader and new leader might produce out-of-order data.

One solution to clock skew is to set up a dedicated synchronization service called an NTP service. However, this approach is not perfect — since it involves network operations, some error is inevitable. When relying on NTP to solve clock skew, the system design must tolerate very small timing errors.

Actually, beyond forcibly aligning clocks, there are simpler approaches to consider. First, let’s think about a question: do we really need to ensure messages are ordered by real-world physical time? Not really. What we need is a consistent, reproducible way to determine message order so that all nodes agree on the order. In other words, messages don’t have to follow physical chronological order, but the order determined by different nodes should be the same.

The Lamport Clock technique achieves this goal. Its logic is simple, as shown in the diagram:

Operations on a local node increment the local stamp by 1. During network communication — for example, when node C receives data from node B — it compares its current stamp with B’s stamp + 1 and takes the larger value as its new stamp. This simple operation guarantees that the order of any two causally related operations (whether on the same node or communicated between nodes) is consistent across all nodes.

Design Principles Summary

Additionally, there are relatively simpler considerations that frequently arise in distributed system design — such as how to distribute data evenly across nodes. For this, we might need to find an appropriate shard key based on the business scenario, or find a suitable hash algorithm. There is also consistent hashing technology, which gives us more flexibility in controlling data distribution.

Another key consideration in distributed system design is how to measure system performance, with metrics including latency, throughput, availability, consistency, and scalability. These are easy to understand conceptually, but measuring them more comprehensively — especially observing them conveniently — is itself a large topic.

Source: https://lichuanyang.top/en/posts/45718/

First, let’s sort out the necessary steps in the entire development process. I will briefly list them, including project creation, development, code review, deployment of test environment, deployment of grayscale and online environments, viewing online monitoring, and logs. , and troubleshooting problems, etc.

Based on this entire process chain, let’s take a look at what aspects we need to ensure in order to improve development and operation and maintenance efficiency as much as possible, and what low-cost solutions can be found in these aspects.

There are a few points:

• Quickly create a project and add necessary web, monitoring, log and other components;
• Improve the development efficiency of some conventional code development, such as database addition, deletion, modification and query;
• Simplify the process from packaging to deployment, and the test environment can be deployed automatically;
• A complete set of monitoring and alarms can be easily added to each service, including common machine, JVM, interface level indicators, and business-customized indicators;
• There is a place to centrally view the logs of each node;
• When necessary, powerful tools can be used on online nodes to help troubleshoot problems.

Next, let’s take a look at how to solve these problems respectively.

Deployment environment

First of all, in the title of this article, it is actually assumed that the deployment environment is kubernetes, so let me first talk about why kubernetes is the only option. Just flip through any introduction to kubernetes, and we can see many explanations of its advantages, so we won’t go into details here. The most critical one, and the benefit that everyone can experience personally, is that it provides a complete automatic expansion and contraction mechanism. It is very simple to set an expected value for CPU consumption, and the k8s cluster can adjust the number of pods based on the current CPU load. I compared the data before and after we enabled automatic expansion and contraction. The daily machine cost can differ by nearly 600 US dollars, accounting for more than 1/3 of the total cost.

Since the operation and maintenance cost of k8s itself is very high, it is recommended to directly purchase the services of a cloud service provider. Both AWS and Alibaba Cloud provide very complete k8s cluster functions.

In addition, it is recommended to use kuboard (https://kuboard.cn/) this tool to manage k8s. kuboard is a graphical k8s management tool, including common operations such as deployment, configuration, expansion, and logging in to pods, all of which can be operated on a graphical interface and is very convenient to use.

Project Development

Generally, you can create a project based on spring initializr. For common configurations such as logging and monitoring, it is recommended to compile a list of best practices and create a template project. New projects can be created based on this template project, using mvn archetype or similar tools to make the process smoother.

As for some commonly used basic codes such as databases and caches, I feel that Spring Family Bucket is easy enough to use.

For code review, we can use GitLab’s webhook to automatically send notifications to the project team when the code is submitted.

Packaging & Deployment

For springboot projects, you can use buildpacks to create docker images without having to consider the details of the docker file. For specific buildpacks, we use paketo buildpacks. Others such as cloudfoundry and heroku are similar. Currently, we have not studied the specific requirements for these packages.

Next, we can still use gitlab’s webhook to trigger a Jenkins build task. In the Jenkins build task, we can complete the operations of packaging and deploying to the kubernetes test cluster.

Monitoring & Alarming

It is recommended to use the prometheus + grafana package. Regarding the use of prometheus and the configuration under the springboot project, you can refer to my previous article h[ttps://lichuanyang.top/posts/28288/](https://lichuanyang. top/posts/28288/) .

On k8s, you can install a weave cloud agent, and then configure the automatic crawling of the prometheus interface.

In grafana, you can directly write promeql configuration monitoring reports. In addition, on the Grafana official website, there are a large number of charts shared by others that can be used directly.

In grafana, you can also configure alarms with various custom rules. If you use Feishu, you can easily use Feishu as the alarm channel for grafana by configuring grafana assistant in Feishu.

log

You can use loki (https://grafana.com/oss/loki/) as a tool for log collection and query. Loki can be considered as a lightweight ELK, and its maintenance cost will be much lower than ELK.

Troubleshooting

For java projects, using the artifact arthas can solve most problem troubleshooting needs. For arthas access, you can use arthas springboot starter. For pods on k8s, if the environment can be connected to the office environment, you can use the port forward function of k8s to forward the arthas port to the local. Of course, if you do this, be sure to control permissions.

Canary deployment

Regarding the role and implementation of canary deployment, you can refer to my other article https://lichuanyang.top/posts/30764/

To sum up, for operation and maintenance, you only need to maintain some infrastructure such as gitlab, kuboard, prometheus, grafana, loki, etc., and they are basically tools that are relatively simple to maintain. On this basis, with the help of reasonable processes and techniques, we can achieve a very good development experience.

Original address: https://lichuanyang.top/en/posts/40964/

Career Review

The biggest change in 2021 came from work. Since the industry I was in disappeared from the spotlight overnight, my plans to develop at one company for the long haul fell through, and I was forced to restart my career planning. A key consideration was whether to join a big company or a small company. The reason I mentioned reaching a critical crossroads is that at this age, whichever path I choose, there’s essentially no turning back. The capability requirements for a senior employee at big companies versus small companies are vastly different. At this stage, you no longer have the freedom to jump between big and small companies like you could in your early career after graduation.

To make this decision, I asked myself several questions:

Would I grow faster personally at a big company compared to a small one?

Would life at a big company be more comfortable than at a small one?

Will I need another big company experience as credentials in the future?

Could I earn more income at a big company?

After careful consideration, the answer to all these questions was actually no. Personal growth depends on the individual — I’m fundamentally not someone easily influenced by my environment. Regarding work-life balance, in theory it shouldn’t have much to do with company size, it just happens that several big companies in China aren’t great at this. On the credentials front, since I already have big tech experience, plus my educational background and other prior experience, I don’t think I’d lose out in that regard. Also, based on this job search and my experience interviewing candidates over the past two years, I’ve come to believe that the substance of your projects matters more than which company you were at. As for income, it depends on the macro environment, the company’s trajectory, the boss’s style, and luck — it generally doesn’t correlate much with company size.

After all this deliberation, I realized that the only advantage of joining a big tech company was that it would look good on paper. So I made my decision: completely abandon the big company idea and instead seek out promising startup companies with good culture and outlooks. Speaking of which, I’m sometimes deeply grateful for what seems like fate’s grace — it’s as if the universe always presents me with suitable options at key moments. In reality, though, I mainly have to thank the headhunter who provided information and support throughout the process, leading me to a new company that satisfied me on every front. I’ve been at the new company for almost four months now, and during this period, the room for doing meaningful work, personal growth, WLB, and financial returns have all exceeded expectations.

Life Changes

With the new work schedule, I also have more time with my daughter. The little girl has recently entered a language explosion phase, saying new words almost every day, and my sense of life happiness has reached a very high level.

Technical Growth

Additionally, I set some personal growth goals at the beginning of each year. At the start of 2021, my key goals were to expand and solidify my capability circle — striving to master new skills and reinforce existing ones. Specifically, this included improving technical and management skills related to my day job, enhancing business acumen and investment ability, and building some foundational skills.

On the professional front, I read Zhou Zhiming’s “Phoenix Architecture,” which played a key role in helping me build a more complete knowledge system and fill in some knowledge gaps. I studied Wang Zheng’s design patterns column on Geek Time, gained deeper understanding of design patterns, and gave a design patterns talk within my company. I also did a concentrated study of Elasticsearch, a technology I’d been using for a long time without seriously understanding.

On management, I learned some theoretical knowledge this year and have been working on applying it to practical work, with decent results.

On investments, I actually made some very mistaken attempts this year. Starting from early in the year, I tried to analyze company fundamentals myself to find suitable investment targets, reading many books and studying the experiences of notable investors. My current conclusion is that the barrier to this is extremely high — it’s unrealistic to achieve professional-level investment results through self-study alone. Based on this experience, my investment returns turned negative for the first time this year. Going forward, I’ll abandon individual stock investing and shift entirely to fund and index dollar-cost averaging. However, I won’t stop learning about business and investment knowledge, because I’ve realized that this knowledge can meaningfully transform one’s way of thinking, and the gains are well worth it.

Regarding foundational capabilities, I focused this year on cultivating what I call “clarity power.” This concept comes from the book “Cognitive Awakening” — the idea being that our fears are often rooted in “ambiguity,” meaning we haven’t clearly figured out what we’re actually facing. Once we have a clear understanding of a situation, we realize it’s not that difficult. The first time I encountered this concept, I immediately recognized that I’d always had a significant problem in this area, so I began deliberate practice. The results have been quite good — whether in work or life matters, I feel much more capable of handling things smoothly.

2022 Outlook

In summary, this year has been a year of critical decisions for me, and I’m steadily moving toward my life goals. I’m grateful to everyone who has helped me along the way. Here’s to continuing to give it my all in 2022.

Source: https://lichuanyang.top/en/posts/2345/

Frequently Asked Questions

Q: What’s the path forward for mid-career engineers?

The path doesn’t depend on company size — it depends on your capability circle. At the mid-to-late career stage, what matters more than “which company to join” is: what domain you have unique insights in, what level of technical problems you can solve, and whether your experience has crystallized into transferable methodologies. The author’s choice — letting go of big-tech prestige to seek startups with good culture and promising prospects — offers one viable path. But it’s not the only one. The key is to actively think about what you want, not passively wait for opportunities to land.

Q: How should technical people respond to industry changes?

In one phrase: expand your capability circle. An industry can vanish overnight (as the tutoring industry did for the author), but if you’ve been consistently broadening your capabilities, you gain the mobility to move across industries. Concretely: maintain technical sharpness (read seminal books like “Phoenix Architecture”), cultivate transferable foundational skills (like “clarity power”), and never anchor your sense of professional security to a single company or industry.

Q: Big company or startup — which is better for senior engineers?

It depends on what you need most at this stage. Big companies offer platform credentials and resource leverage, but personal growth pace and work-life balance aren’t guaranteed. Startups offer broader scope and faster decision-making, but stability is lower. The author’s conclusion after comparison: personal growth comes from yourself, not company size; if you already have big-tech experience, credentials matter less; and income depends more on the macro environment and luck than company size. The core advice: clarify your current-stage selection criteria before deciding, because at this stage of your career, there’s rarely a path back.

Design Thinking

An IP address is essentially a 32-bit integer. An “IP range” is simply a continuous range of integers. Given an IP, we need to find which range it falls into — a classic interval search problem.

The most direct approach is to iterate through all IP ranges — O(n) complexity. But there are hundreds of thousands of IP ranges globally, so iteration is too slow. A better approach is to sort the starting addresses of the IP ranges into an array and use binary search — O(log n) complexity, with only about 19 comparisons needed for hundreds of thousands of ranges.

Data Source

IP address database data can be obtained for free from IP2Location Lite. Sample data format:

1
2
3

16781312,JP
16785408,CN
16793600,JP

Each line contains two fields: the IP range’s starting address (converted to an integer) and the country code.

The original data also includes the end address of each IP range, but IP2Location’s data segments are continuous and gap-free — the end address of one range is exactly the starting address of the next range minus one. Therefore, we can store only the starting addresses, with the end address naturally determined by the next range’s starting address. This optimization saves half the memory.

Data Structure

1
2
3
4
5
6
7
8

public class IpCountryLookup {
    // IP range starting addresses (sorted)
    private final long[] startIps;
    // Corresponding country codes
    private final String[] countryCodes;
    // Total number of ranges
    private final int size;
}

Two parallel arrays: startIps[i] and countryCodes[i] correspond one-to-one. The binary search is performed on startIps, and once found, the result is taken from countryCodes.

Core Implementation: Binary Search

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

public String lookup(String ip) {
    long ipLong = ipToLong(ip);
    
    // Binary search: find the last starting address <= ipLong
    int left = 0, right = size - 1;
    while (left <= right) {
        int mid = left + (right - left) / 2;
        if (startIps[mid] <= ipLong) {
            left = mid + 1;
        } else {
            right = mid - 1;
        }
    }
    
    // 'right' is the position of the last element <= ipLong
    if (right >= 0) {
        return countryCodes[right];
    }
    return "Unknown";
}

// Convert IP string to long
private long ipToLong(String ip) {
    String[] parts = ip.split("\\.");
    return (Long.parseLong(parts[0]) << 24)
         + (Long.parseLong(parts[1]) << 16)
         + (Long.parseLong(parts[2]) << 8)
         + Long.parseLong(parts[3]);
}

Performance Data

• Memory Usage: ~300,000 IP ranges, two arrays with ~300K elements each, total about 4 MB
• Query Latency: < 10 microseconds per query (~19 comparisons for binary search)
• QPS: Easily handles millions of queries per second on a single machine
• Data Updates: IP database updates roughly once a month; a scheduled task automatically pulls and reloads

Other Data Source Comparisons

If you only need country-level granularity, IP2Location Lite is sufficient. For city-level accuracy, MaxMind GeoLite2 is recommended.

Deployment

The project is built on Spring Boot and supports two deployment methods:

1
2
3
4
5

# Run directly
mvn spring-boot:run

# Docker
docker run -p 8080:8080 lcy362/ip-country

Try it online: http://ip-country.lichuanyang.top/

Project source: https://github.com/lcy362/ip-country

Source: https://lichuanyang.top/en/posts/36780/

In a typical development workflow, there are various review stages — product spec reviews, design reviews, test case reviews, code reviews, and so on. To conduct these reviews, meetings are inevitably introduced. Some people might think these activities are useless and a waste of time. In reality, they are extremely important. If you have the impression that reviews don’t matter, it’s likely due to one of two reasons: either a lack of understanding, or poor review practices in your environment.

Value of Review for the Team

Regarding the value of reviews, the most significant impact for a team is raising the team’s baseline. Imagine if no reviews were conducted at any stage — a product spec is handed off, developers just start coding, and any individual oversight could bring endless risks to the entire project. Multiple review stages essentially describe a business solution in different forms — product specs, code, test cases — and then involve more people to inspect each form, preventing individual oversights from creating business risks.

Furthermore, the focus of this article is to discuss the value of participating in reviews for an individual, beyond the benefits it brings to the team.

Value of Review for the Reviewee

First, for the person being reviewed — the author of the product spec, code, or other deliverables — a review is essentially a process where others help you improve. It’s a rare opportunity to receive direct feedback. For many people, compared to their student days, the biggest challenge of learning during their career is that no one “grades” their homework. After reading books or watching videos, there’s no way to confirm whether they’ve truly understood the material. Timely feedback, regardless of what you’re learning, is crucial. Among the reviewers will be your superiors and more capable colleagues. If they can thoroughly understand your solution or code, they can certainly provide valuable feedback. A review essentially gives you the opportunity to receive their feedback.

This implicitly requires the submitter to express their solution clearly, so others can participate in the review with minimal effort. For example, when writing code, use appropriate design patterns and write readable code. When writing documents, have a well-organized layout with proper sections and highlight key points. When presenting a solution to others, consider the audience’s knowledge background to ensure they can understand your proposal.

Another important aspect of reviews for the submitter is risk and responsibility sharing. Regardless of your company’s policies or your position, when a project you’re responsible for doesn’t go well, a significant portion of the consequences will ultimately fall on you. This may come as direct performance penalties, or as others questioning your capabilities — either way, it’s something nobody wants to face. Reviews transfer some of these consequences to the entire team. Of course, don’t think too much — the primary responsibility still remains with the submitter. This again highlights the importance of clear expression. The submitter needs to ensure that the key points of their solution are effectively communicated to the reviewers. If the reviewers successfully absorb this information, have thorough discussions during the review, and reach a consensus, that part of the decision can be considered as jointly made by the team.

Value of Review for the Reviewer

Now, let’s look at the value for reviewers — what does actively participating in others’ reviews offer? The primary benefit is a learning opportunity. As the saying goes, “When three people walk together, there must be one who can teach me.” From others’ solutions, you can certainly discover excellent points worth learning. When you find that someone else’s approach differs from your own expectations, it’s an opportunity for deep thinking — you can continuously question yourself to determine which approach is better until you reach an irrefutable conclusion. This kind of deep thinking significantly improves your own thinking methods.

Additionally, participating in reviews is also a way to rapidly gain experience. Reviewing is essentially spending review time to achieve an effect nearly identical to doing the work yourself. If you review thoroughly enough, you’ll gain a comprehensive understanding of how something is done, and this experience can fully count as part of your project experience.

Summary

This article comes to an end here. I hope you’ve gained an understanding of why reviews should be done and how to conduct them. Feel free to reach out to me through my WeChat official account (Mobility) or personal website.

Source: https://lichuanyang.top/en/posts/57205/

Trade-offs Across Four Dimensions

Career choice ultimately comes down to two key decisions: city and company. Choosing a city is highly subjective. For example, as someone from the north, I don’t like the climate in southern cities, I want to be close to home, and I need the opportunities that a big city offers — so Beijing was almost the only option. As practical advice, I’d suggest considering fast-growing strong second-tier cities like Chongqing or Hefei, where you can ride the wave of urban development and increase your own value along the way.

Within the scope of Beijing, I’ll discuss company choices based on my own experience.

Hukou: The Ticket to a Tier-1 City

When it comes to Beijing, one topic that inevitably comes up is the hukou (household registration). However, I strongly advise against letting a hukou decision dictate your entire life trajectory — for instance, joining a state-owned enterprise just for a hukou even though you have no desire to work in the system. Always prioritize thinking through your career development direction before worrying about hukou.

Many people agonize over public sector vs. private sector, big company vs. small company — but these are all surface-level choices. Even within the public sector, experiences can vary enormously. The right approach is to first clarify what you truly want, then seek out suitable opportunities. Knowing what you truly want isn’t easy for everyone, so let me break it down. Consider these questions:

• What do you most want from work? Money? Power? A sense of inner achievement? Or do you not expect anything from work at all?
• How much of your life do you want work to occupy?
• Among the resources gained through work (connections, information, etc.) and the capabilities developed, which do you feel more confident carrying into a different environment?
• Do you prefer doing routine, administrative work, or do you want to continuously tackle challenging problems?

For any of these questions, no answer is inherently better than another. Tackling difficult problems isn’t superior to doing routine work — it’s simply a choice based on personal values.

With any combination of these choices, you can find suitable opportunities. For instance, if you don’t expect much from work, don’t want it to dominate your life, and are willing to do routine work, some peripheral state-owned institutions could be a good fit. If you want power, are willing to let work take up significant time, and prefer to accumulate resources through work, consider core state-owned institutions with real authority. And if you want to earn money and prefer to develop personal capabilities, being a programmer is a solid choice.

Once you’ve thought through these questions, you can choose a company. Many times, we may not find a place that satisfies all our requirements. That’s okay — we can use job-hopping to address one part at a time, continuously gather information and improve our abilities, and find opportunities when we’re ready to seize them.

Big Tech: The Value and Cost of the Platform

Let me share how I approached my own key decisions based on my experience.

When I first graduated, my goal was to get a Beijing hukou while staying in the internet industry. I had two broad directions: competing for an SSP offer at an internet company — but my abilities at the time weren’t sufficient, so that path was hard to take. The other path was to find opportunities within state-owned enterprises where the business closely resembled internet scenarios, ideally consumer-facing products with large user bases and data volumes, so I wouldn’t fall too far behind industry-leading technologies. Fortunately, I found such an opportunity and successfully seized it. After joining, it was pretty much as I expected — although the company’s technical level wasn’t high and the corporate culture had many issues, the business scenarios were excellent and I had plenty of room to contribute. Through my work, I learned a great deal.

Next, I needed to address my big tech experience, so when I changed jobs, I only considered top-tier tech companies. The work itself was nothing special to speak of — although I wasn’t happy during that period, I achieved the goals I’d set, so there are no regrets.

For my next job change, I had three goals: significantly increase my income to close the gap with peers caused by my time at the state-owned enterprise; not work too much overtime; and have more relaxed interpersonal relationships. These three goals seem contradictory, don’t they? But with effort, I always managed to find suitable opportunities, and that’s how I ended up at my current company. Not all problems were solved perfectly, but they were addressed enough to make my work experience quite enjoyable.

Through my own experience, you can see that I always seemed to make contradictory demands — wanting both fish and bear paw — and fortunately, I managed to get both in the end. The truth is, there are so many companies in the world that no two conditions are necessarily mutually contradictory. There’s no need to give things up prematurely. For example, does being a programmer necessarily mean 996 work and giving up your life? Obviously not.

I’d like to address two more specific questions that arise from the above.

First, whether to choose a Beijing hukou. Let me first say what I gained from it. I caught the wave of property price surges in 2015-2016, which significantly increased my family’s assets even though I had no financial planning awareness at the time. This no longer holds true today — under the policy of housing for living, not speculation, even Beijing properties can’t outperform the stock market. Second, some procedures became slightly more convenient to handle, which I consider negligible — I probably don’t deal with paperwork more than once a year, and an increasing number of procedures support remote processing. Third, I can rest assured about my children’s education in Beijing without worrying about future uncertainty. I say “uncertainty” because I think it’s highly uncertain whether non-Beijing hukou holders will still be barred from taking the gaokao in Beijing nearly twenty years from now, and equally uncertain how competitive Beijing’s gaokao will become. A Beijing hukou essentially fixes my future choices, but what those choices will actually amount to is hard to predict.

So currently, the value of a Beijing hukou is quite limited. If I could go back to 2015, I’d still choose a Beijing hukou without hesitation. But if I had to make this choice today in 2021, I’d most likely decline it.

Second, regarding big tech experience. My view is that you must have it, and the sooner the better, but there’s no need to stay at a big company for life. The value of big tech experience has been thoroughly discussed by many, so I won’t repeat it here. Feel free to reach out if you’re interested in discussing further.

How to Choose

The above covers most of this article. One point I haven’t mentioned is the midlife crisis — some might wonder why. The reason is that I don’t think it’s a problem that can be solved through career choices alone. Some people think the public sector is stable — even if you coast every day, no one can touch you. But I don’t consider stability that depends on external conditions to be true stability. The public sector has had problems before in history. This path might give you a 99.99% chance of living peacefully for life, but if that 0.01% scenario occurs, what will you do? Being a programmer is different — perhaps only a 50% chance of staying comfortably at one company, but the capabilities developed through work give me the confidence to handle the other 50%. The only way to address the midlife crisis is self-improvement — whether building capabilities or accumulating resources, you must gain something from work to maintain your competitiveness.

I’m Liusha. You can connect with me through the WeChat public account (Mobility), my personal website, and other channels.

Source: https://lichuanyang.top/en/posts/34931/

Regarding the inputs and outputs of RPC interfaces, there has long been a school of thought that advocates wrapping all inputs and outputs into individual request and response classes. In this article, we will analyze whether this approach constitutes a sound design.

Input Parameter Encapsulation Analysis

Let’s start with inputs. I’ll state the conclusion first: encapsulation should only be done when there are excessive input parameters. In all other cases, there is absolutely no justification for encapsulation.

The reasons given for wrapping input parameters into a request class typically boil down to the following:

• Maintaining compatibility when interface parameters are modified
• Defining a generic abstract request class for unified management

First, the first point is a complete fallacy. The first time I encountered this claim, I was almost misled by it as well. But upon careful reflection, it becomes clear that this argument doesn’t hold at all.

Even with a request body approach, if the interface provider adds a mandatory field, the caller still needs to include this parameter to make the call. Compared to passing parameters directly, this doesn’t solve any problem. Furthermore, this kind of issue is fundamentally unsolvable because such situations should not arise in the first place. Interface providers have an obligation to ensure that subsequent modifications are forward-compatible. As for adding non-mandatory fields, even without wrapping in a request class, you can implement the new interface and maintain backward compatibility through an adapter pattern, and later asynchronously deprecate the old interface when the opportunity arises.

The second point does have some merit.

• For example, adding unified request parameters in an abstract class. I believe this is better handled in the framework layer rather than being exposed in business code.
• Another example is adding unified parameter validation. But if that’s the only purpose, using an overly abstract class is inappropriate. Instead, each related business domain should have its own abstract class, since only that domain would have such requirements. Moreover, even without wrapping into a request class, such requirements are not difficult to implement, as the difficulty itself doesn’t lie in non-uniform input structures.

The biggest problem with wrapping input parameters into a request class is that it destroys code readability and creates semantic ambiguity. A method like queryById(long id) has very clear semantics — a glance at the interface definition reveals the inputs and outputs. But after forced encapsulation, what inputs the interface actually requires becomes unclear. Additionally, if a request contains three parameters, can any combination of them return a result? This is also something you cannot determine just by looking at the interface. With normal parameter definitions, you can define a series of interfaces using the adapter pattern, making it easier for callers to understand.

The only condition for encapsulating input parameters is when there are too many parameters, and you need to use other means such as comments or documentation to specify the legal input values and upgrade strategies for the interface.

Output Result Encapsulation Analysis

Now let’s discuss outputs. Wrapping output results into a response body is somewhat more reasonable than wrapping input parameters, but it still shouldn’t be used indiscriminately. The general purpose of wrapping output results is to add a return status code layer.

Interface Segregation Principle Perspective

This essentially forces callers to understand the error conditions of the interface. We know that one of the fundamental design principles is the “Interface Segregation Principle,” which states that clients should not be forced to depend upon interfaces that they do not use. The situation here is analogous: if callers genuinely don’t care about the reasons for request success or failure, they shouldn’t be forced to understand them. For example, if I just want to query a count, there are only two possibilities — found or not found — and I don’t want to care about why it wasn’t found.

Therefore, when designing interfaces, you need to carefully consider whether the error conditions of an interface should be information that is provided externally. Only on this basis can you determine whether wrapping in a response object is appropriate or not.

Conclusion and Recommendations

In summary, blindly wrapping all input and output parameters is never advisable. You must carefully consider when encapsulation is appropriate and when it is not. If you have any thoughts on this topic, feel free to discuss them with me.

Source: https://lichuanyang.top/en/posts/20888/

Why I Don’t Do Infrastructure Development

First, I want to address a common misconception: that infrastructure development makes it easier to learn and grow faster. I believe this is nothing more than wishful thinking by business developers. Infrastructure development isn’t as conducive to learning as people think, and business development isn’t as detrimental to learning as people think — I’ll elaborate on the reasons below. The reason many people hold this belief is largely because infrastructure development positions typically have higher hiring standards, so these individuals naturally have better learning ability and motivation.

In reality, the daily work of infrastructure development isn’t that different from business development — both involve working on requirements. The only difference is that business requirements come from product managers, while infrastructure requirements come from other developers, or from the infrastructure developer’s own thinking and ideas. This brings me to my first reason for not wanting to do infrastructure development: the source of requirements is unscientific. If you only do infrastructure development and are detached from actual business scenarios, your understanding of technology is likely to be biased, and things you want to build may just be self-indulgent exercises with no practical application value. Even when collecting requirements from business developers, there are similar issues — their suggestions may reflect only shallow, unconsidered thoughts. Moreover, business developers may not accurately understand what infrastructure components should actually handle; they may not distinguish between what should be solved by infrastructure and what they should handle themselves.

This brings me to my second reason for not wanting to do infrastructure development: unclear boundaries of responsibility. This is closely tied to the capability of the business teams you work with. Sometimes, when something on your end isn’t done well, the business team finds a way to work around it. Other times, even when something has nothing to do with you, you’re forced to deal with some upper-layer logic.

As you can see, both issues above are extremely dependent on colleagues’ abilities. So if you insist on doing infrastructure development, you must go to a company with strong technical talent — this directly affects your work happiness. The company choice also impacts another aspect: the nature of the work. Simply put, it’s a difference between building wheels versus improving open-source tools — these two types of work are worlds apart. Building wheels requires deep familiarity with various fundamental knowledge and can indeed promote growth — this is the origin of the idea that “infrastructure development helps you grow.” But this kind of work only exists at a handful of big companies. The vast majority of infrastructure work — improving open-source tools — is no different from business development. You only need to learn the ins and outs of the open-source tools to complete most of the work, with neither depth nor breadth. This is why I said earlier that infrastructure development doesn’t help with learning.

So, to do infrastructure development, company selection is crucial — it must be large AND technically strong; both conditions are essential. You might ask: why not just go to such a company? That’s fair, but it brings its own problem — my third reason for not wanting to do infrastructure development: the career choice is extremely narrow. In China, there are only two companies doing Java infrastructure well, and everyone knows what their corporate cultures are like. So if you don’t want to go to those companies, does that mean you can’t do it at all? Even if you’re willing, the massive supply-demand imbalance would lead to insane involution, similar to algorithm positions. Do you have enough confidence to win that game? In contrast, for business development, big tech experience is just a necessary learning experience and a highlight on your resume. Once you’re familiar with the entire standardized workflow at a big company, you can freely choose wherever you want to go.

The Value of Business Development

I’ve talked too much about infrastructure development above. Let me now get to the main topic: the value of business development. In my view, business development has improved my capabilities primarily in three areas: accumulation of business knowledge, expansion of technical breadth, and improvement of comprehensive abilities.

First, business knowledge — business knowledge is essentially an understanding of the real world. For me personally, learning about how different industries operate in my spare time is first and foremost an interesting thing to do. Second, continuously learning industry knowledge across different domains has noticeably improved my cognitive abilities. Can this ability be monetized? Of course. For example, when investing in stocks, the accumulation from work has given me the ability to analyze a company’s business model — I can clearly understand what makes a good company and a good stock at a deeper level than most people. Of course, I’m getting ahead of myself here — take it as food for thought. The key is to follow your interest and see if you’re willing to learn about things beyond technology.

Next, technical breadth. The biggest difference between business development and infrastructure development is that infrastructure development demands more depth, while business development demands more breadth. Business development requires familiarity with a wide variety of frameworks and middleware — not just surface-level knowledge, but enough depth to understand what problems they solve, what problems they introduce, and which scenarios they’re suited for. You need to comprehensively understand complex business scenarios and complex systems, and be able to quickly locate problems when they occur. Beyond knowledge accumulation, this also requires experience gained from real-world scenarios.

Then there’s the improvement of comprehensive abilities. Compared to infrastructure development, business development exercises a person’s general foundational capabilities more — communication, problem analysis and problem solving, and the ability to weigh trade-offs. These abilities are related to technology, but not so closely. To be extreme, even if you change careers in the future, these universal capabilities will be very helpful. I often think about what the difference is between learning to use an open-source tool and playing Honor of Kings. As a beginner, following an example to build a demo is like encountering a game for the first time and just playing around. As you get more familiar, some people become emotionless tool-switching machines, while others gradually explore the underlying implementation. Just like in gaming — some people play thousands of matches without ever understanding the difference between Lightning Dagger and Infinity Edge, while others explore the numerical mechanics of damage. So I think we shouldn’t view programmers as some uniquely special profession — it’s fundamentally no different from other professions. Most professions are just solving problems in different ways: lawyers rely on law, fund managers rely on understanding markets and fundamentals, and programmers rely on code. What we continuously develop through work is our problem-solving ability, and no one requires the solution to be limited to technology alone.

In business development, we must continuously accumulate capabilities in these three areas, so that we can build momentum over time and confidently face the midlife crisis. Of course, I’m not writing this to say that business development is inherently better — I mainly want to help everyone clearly analyze the differences between business development and infrastructure development, so you can make a rational choice.

If you have any thoughts, feel free to reach out and discuss with me.

Source: https://lichuanyang.top/en/posts/40071/

Prerequisites

When we mention design patterns, the first thing to understand clearly is object-oriented thinking. I believe that even if you can’t describe it very precisely, you should already be fairly familiar with object-oriented programming.

Let me quickly review — object-oriented programming has four fundamental characteristics: encapsulation, abstraction, inheritance, and polymorphism.

Encapsulation: Exposes only limited interfaces, authorizing external access. Centralizes logic, making it more controllable; improves readability and maintainability; also enhances usability.

Abstraction: Hides the specific implementation of methods, so that callers only need to care about what functionality the method provides, without needing to know how those functions are implemented.

Inheritance: The benefit is code reuse.

Polymorphism: Subclasses can replace parent classes, improving code extensibility and reusability.

What Are Design Patterns?

Design patterns are a set of practical solutions or design approaches summarized for frequently encountered design problems in software development, based on fundamental design principles.

As you can see, design patterns are very practice-oriented, more concrete and executable compared to design principles.

Therefore, before learning about design patterns, you need to understand some basic design principles. These principles are the key to guiding us to write good code.

So, what is good code?

What Makes Good Code

This is actually quite hard to describe. We can try to summarize some characteristics of good code, such as:

Maintainable, extensible, readable, testable, reusable, and concise.

To achieve these standards, we need some basic design principles first.

Fundamental Design Principles

SOLID

Single Responsibility Principle

The description is simple: a class should only be responsible for one task or function. But actually doing this well is quite difficult.

To determine what counts as a single responsibility, we can set some simple criteria, such as: too many lines of code; too many dependent classes; too many private methods; difficulty naming the class; too many methods in the class with concentrated functionality. However, there’s no universal standard — you must consider the actual situation.

For example, consider this class:

1
2
3
4
5
6
7
8
9
10
11

Student {
    int id;
String name;
String age;
...
String province;
String city;
String county;
String detailAddress;
...
}

There are at least four fields related to addresses. Should we extract the address-related responsibilities into a separate Address class?

Actually, it depends on the situation. If the address properties are just pure display information, then placing them directly in the Student class is fine. But if there’s a complex logistics system behind them — meaning lots of complex logic related to addresses — then that part should be extracted separately.

In practice, we can start with a coarse-grained class to meet business requirements. As the business develops, if the coarse-grained class becomes increasingly large with more and more code, we can then split it into several finer-grained classes. This is called continuous refactoring.

Open-Closed Principle

Open to extension, closed to modification.

Adding a new feature should be done by extending code on top of existing code (adding modules, classes, methods, properties, etc.), rather than modifying existing code (changing modules, classes, methods, properties, etc.).

The Open-Closed Principle is one of the harder principles to understand.

Questions like “What kind of code change is considered an ‘extension’? What kind of code change is considered a ‘modification’? How do we determine if the Open-Closed Principle is satisfied or violated? Does modifying code always mean violating the Open-Closed Principle?” are all difficult to answer.

Yet this principle is one of the most useful, because extensibility is one of the most important metrics for code quality.

In practice, we don’t need to agonize over whether a code change is a “modification” or an “extension,” nor worry too much about whether it violates the Open-Closed Principle. We should return to its original intent: as long as the existing code’s behavior isn’t broken, the Open-Closed Principle isn’t violated. A simple criterion is: when functionality hasn’t changed, if the corresponding unit tests aren’t broken, the change satisfies the Open-Closed Principle.

How to achieve openness to extension while being closed to modification also requires gradual learning and accumulation. Among the 23 classic design patterns, most exist to solve code extensibility problems, primarily following the Open-Closed Principle.

When writing code, you can assume changes won’t occur; when changes do arise, abstraction is needed to isolate future similar changes.

Liskov Substitution Principle

Subclasses should be able to replace parent classes anywhere, while ensuring the original program’s logical behavior remains unchanged and correctness isn’t compromised.

The Liskov Substitution Principle has another more practical and instructive description: “Design By Contract.” In this description, the relationship between subclasses and parent classes can be replaced with the relationship between interfaces and implementations. In other words, implementation classes cannot violate the functionality, inputs, outputs, exceptions, etc. that the interface declares it will provide.

Interface Segregation Principle

Clients should not be forced to depend upon interfaces that they do not use.

The core element is that an “interface” should have as focused a responsibility as possible. Callers should only depend on the parts they need, rather than depending on or calling the entire “interface.”

Here, “interface” can refer to different meanings — for example, a set of API interfaces (i.e., a proxy we use).

For instance, in a project that handles business processes, generates data, and then needs to provide that data as a base service, you would provide two interfaces: one for processing internal processes and one as a base service for external access. This approach can help avoid accidental misuse to some extent.

You can also understand an interface as a single interface method, in which case it means each method should have a single responsibility.

The Interface Segregation Principle is somewhat similar to the Single Responsibility Principle, but there are slight differences. The Single Responsibility Principle targets the design of modules, classes, and interfaces. The Interface Segregation Principle, compared to the Single Responsibility Principle, focuses more on interface design. It provides a criterion for judging whether an interface has a single responsibility: indirectly determined by how callers use the interface. If callers only use part of the interface or some of its functionality, the interface design lacks a single responsibility.

Dependency Inversion Principle

High-level modules shouldn’t depend on low-level modules. Both modules should depend on abstractions. In addition, abstractions shouldn’t depend on details. Details depend on abstractions.

The division between high-level and low-level modules simply means that in the call chain, the caller belongs to the high level, and the callee belongs to the low level.

This principle is more targeted at framework development. In business development, high-level modules depending on low-level modules is actually perfectly fine.

Taking Tomcat as an example: Tomcat calls web programs to run, so Tomcat is the high-level module.

Tomcat and application code don’t have direct dependencies on each other; both depend on the same “abstraction,” which is the Servlet specification. The Servlet specification doesn’t depend on specific Tomcat container or application implementation details, while the Tomcat container and application depend on the Servlet specification.

KISS

Keep It Simple and Stupid.

Keep It Short and Simple.

Keep It Simple and Straightforward.

The KISS principle has different formulations, but they all convey roughly the same meaning: keep code as simple as possible.

“Simplicity” is actually very subjective. Code review is a good indirect way to evaluate whether code is simple. If colleagues reviewing your code find many parts hard to understand, the code is likely not simple enough.

Some simple tips: Don’t reinvent the wheel; avoid over-engineering, whether for uncertain future extensibility or minor performance differences.

For example, many methods in StringUtils, in a specific scenario, could likely be implemented with better performance because the general concerns that StringUtils handles don’t need to be considered in specific cases. But should we really do that?

DRY Principle (Don’t Repeat Yourself)

This principle is simple in itself, but note that the “repeat” here is different from simple code duplication. For example, two unrelated methods that happen to have identical implementations are not considered a DRY violation because semantically they’re not duplicating. Conversely, two methods with completely different names and implementations — like isValidIp() and checkIpValid() — that do the same thing are also considered DRY violations.

Also note execution duplication, for example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

void run1() {
a();
b();
}

void run2() {
a();
c();
}

main () {
run1();
run2();
}

Although the code appears well-encapsulated, when the main method is executed, method a is called twice. If a is a method with high overhead, this could be problematic.

Law of Demeter

Each unit should have only limited knowledge about other units: only units “closely” related to the current unit. Or: Each unit should only talk to its friends; Don’t talk to strangers.

“High cohesion, loose coupling” is a relatively universal design concept that can guide the design and development of code at different granularities — systems, modules, classes, and functions.

High cohesion means that related functionality should be placed in the same class (module, system), while unrelated functionality should not be placed in the same class (module, system).

Loose coupling means that in code, the dependency relationships between classes (modules, systems) are simple and clear, depending only on what’s necessary.

For example, qwerty’s previous handling of homework-related logic required understanding the implementation details of each homework type, which was a violation of the Law of Demeter. The correct approach is to define a common model on the homework side, encapsulating the different logic of various homework types within the model, so qwerty only needs to understand this model.

For these design principles, there’s no need to spend too much effort on the exact wording. What’s more important is understanding their underlying ideas — understanding why these principles help us write code that is “maintainable, extensible, readable, testable, reusable, and concise.”

Next, we can talk specifically about design patterns.

Common Design Patterns

There are 23 classic design patterns. With the evolution of programming languages, some design patterns (like Singleton) have become outdated or even anti-patterns, some have been built into programming languages (like Iterator), and some new patterns have emerged (like Monostate).

Creational Patterns (5): Factory Method, Abstract Factory, Singleton, Builder, Prototype.

Structural Patterns (7): Adapter, Decorator, Proxy, Facade, Bridge, Composite, Flyweight.

Behavioral Patterns (11): Strategy, Template Method, Observer, Iterator, Chain of Responsibility, Command, Memento, State, Visitor, Mediator, Interpreter.

Let’s focus on some of the more important ones.

Simple Factory & Factory Method

For example, consider a feature that creates different parsers based on file format to read configuration into RuleConfig. The Simple Factory implementation looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

public class RuleConfigSource {
  public RuleConfig load(String ruleConfigFilePath) {
    String ruleConfigFileExtension = getFileExtension(ruleConfigFilePath);
    IRuleConfigParser parser = RuleConfigParserFactory.createParser(ruleConfigFileExtension);
    if (parser == null) {
      throw new InvalidRuleConfigException(
              "Rule config file format is not supported: " + ruleConfigFilePath);
    }

    String configText = "";
    //从ruleConfigFilePath文件中读取配置文本到configText中
    RuleConfig ruleConfig = parser.parse(configText);
    return ruleConfig;
  }

  private String getFileExtension(String filePath) {
    //...解析文件名获取扩展名，比如rule.json，返回json
    return "json";
  }
}

public class RuleConfigParserFactory {
  public static IRuleConfigParser createParser(String configFormat) {
    IRuleConfigParser parser = null;
    if ("json".equalsIgnoreCase(configFormat)) {
      parser = new JsonRuleConfigParser();
    } else if ("xml".equalsIgnoreCase(configFormat)) {
      parser = new XmlRuleConfigParser();
    } else if ("yaml".equalsIgnoreCase(configFormat)) {
      parser = new YamlRuleConfigParser();
    } else if ("properties".equalsIgnoreCase(configFormat)) {
      parser = new PropertiesRuleConfigParser();
    }
    return parser;
  }
}

This creates a factory class specifically for “producing” parsers.

Placing the parser creation logic directly in the load method would also work. The reason to separate it out is to isolate the creation logic. Therefore, the factory pattern is only used when the creation logic is complex.

The above implementation can be further optimized to eliminate the large number of if-else statements:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

public class RuleConfigSource {
  public RuleConfig load(String ruleConfigFilePath) {
    String ruleConfigFileExtension = getFileExtension(ruleConfigFilePath);

    IRuleConfigParserFactory parserFactory = RuleConfigParserFactoryMap.getParserFactory(ruleConfigFileExtension);
    if (parserFactory == null) {
      throw new InvalidRuleConfigException("Rule config file format is not supported: " + ruleConfigFilePath);
    }
    IRuleConfigParser parser = parserFactory.createParser();

    String configText = "";
    //从ruleConfigFilePath文件中读取配置文本到configText中
    RuleConfig ruleConfig = parser.parse(configText);
    return ruleConfig;
  }

  private String getFileExtension(String filePath) {
    //...解析文件名获取扩展名，比如rule.json，返回json
    return "json";
  }
}

//因为工厂类只包含方法，不包含成员变量，完全可以复用，
//不需要每次都创建新的工厂类对象，所以，简单工厂模式的第二种实现思路更加合适。
public class RuleConfigParserFactoryMap { //工厂的工厂
  private static final Map<String, IRuleConfigParserFactory> cachedFactories = new HashMap<>();

  static {
    cachedFactories.put("json", new JsonRuleConfigParserFactory());
    cachedFactories.put("xml", new XmlRuleConfigParserFactory());
    cachedFactories.put("yaml", new YamlRuleConfigParserFactory());
    cachedFactories.put("properties", new PropertiesRuleConfigParserFactory());
  }

  public static IRuleConfigParserFactory getParserFactory(String type) {
    if (type == null || type.isEmpty()) {
      return null;
    }
    IRuleConfigParserFactory parserFactory = cachedFactories.get(type.toLowerCase());
    return parserFactory;
  }
}

This is the Factory Method Pattern. The Factory Method Pattern essentially creates a simple factory for the factory class — a factory of factories — to create factory class objects.

The principles and implementations of the factory pattern are quite simple. In practice, there are many details worth exploring, but we won’t go into them here. The key point we need to understand is when to use this pattern and what problem it solves.

For the factory pattern, the application scenario is when the creation logic is complex — for example, when there are lots of if-else statements to dynamically decide what object to create; or when creating a single object itself is a complex process and the caller doesn’t need to know how to create it.

Builder

The Builder Pattern is widely used. If you open any project, you’ll find many XXXBuilder classes.

The Builder Pattern is used when a class has multiple configurable options and the user can freely choose which ones to configure.

If you try to achieve this with different constructors, you’d face an endless number of combinations.

And if you use setter methods for configuration, the class completely loses control over its configuration options — callers can invoke setters anytime, anywhere. Additionally, if you need to perform validation after configuration is complete, this approach can’t accommodate that.

The solution for this situation is the Builder Pattern:

1
2
3
4
5
6

ResourcePoolConfig config = new ResourcePoolConfig.Builder()
 .setName("dbconnectionpool") 
 .setMaxTotal(16) 
 .setMaxIdle(10) 
 .setMinIdle(12)
 .build();

The implementation is relatively straightforward.

The factory pattern is used to create a family of related but different objects, while the Builder Pattern is used to customize the same object in different ways.

Singleton

As for what a singleton is and how to implement one — as a classic interview question — I won’t elaborate further. I’ll only discuss the problems with singletons and why they’ve gradually become an anti-pattern.

Problems with singletons:

• Hidden dependencies between classes;
• Dependencies point directly to singleton implementations, violating the principle of depending on interfaces rather than implementations;
• Poor testability: cannot be mocked (in Mockito); global state can cause tests to interfere with each other;
• Doesn’t support parameterized constructors.

However, the advantage of the Singleton Pattern is that in some scenarios, it’s simpler and more convenient to use. So there’s no need to categorically rule out using singletons.

Proxy & Decorator

The Proxy Pattern has the proxy class implement the same interface as the target class, responsible for adding additional logic before and after business code execution (such as performance monitoring, counting, etc.), then delegating to the target class to complete the business function, thereby achieving decoupling between business code and framework code.

The Decorator Pattern achieves functionality enhancement through inheritance of the decorated class. For example, Java’s InputStream series of classes are applications of the Decorator Pattern.

At the implementation level, the Proxy Pattern and Decorator Pattern are almost identical, but their intents differ.

In the Proxy Pattern, the proxy class adds functionality unrelated to the original class, while in the Decorator Pattern, the decorator class adds enhanced functionality related to the original class.

Again, for design patterns, there’s no need to nitpick about what each pattern exactly is. As long as you know such a technique exists and can handle similar scenarios, that’s sufficient.

Adapter

The Adapter Pattern is straightforward — let’s look at an example directly.

1
2
3
4
5
6
7
8
9
10

public void debug(String msg) { 
logger.log(FQCN, Level.DEBUG, msg, null); 
} 

public void debug(String format, Object arg) { 
if (logger.isDebugEnabled()) { 
FormattingTuple ft = MessageFormatter.format(format, arg); 
logger.log(FQCN, Level.DEBUG, ft.getMessage(), ft.getThrowable()); 
} 
}

Application scenarios include:

• Maintaining backward compatibility with old interfaces;
• Handling different inputs;
• Wrapping flawed interface designs;
• …

Facade

The Facade Pattern provides a unified set of interfaces for subsystems, defining a set of high-level interfaces to make subsystems easier to use. This mainly concerns how large an interface’s granularity should be, balancing the generic usability and ease of use of interfaces.

Typically, interfaces with smaller granularity will have better generic usability, but callers would need to invoke multiple methods, which is inevitably less convenient. Conversely, if the interface granularity is too large, there will be insufficient generic usability.

The Facade Pattern solves this problem. I work in online education, and one scenario is displaying various types of questions on the B-side — for example, pre-class questions, in-class questions, and post-class questions. Each category has many specific types, and each type has different data sources.

Here, we need to balance the generic usability and practical usability of interfaces. The Facade Pattern perfectly addresses our problem. First, we define a common model including question stems, reference answers, user responses, etc., then implement a set of interfaces for specific types. On top of that, based on business scenarios, we encapsulate some facade layers that simply combine data from the subsystems.

Strategy

Define a family of algorithm classes, encapsulate each algorithm separately, and allow them to be interchangeable.

The implementation involves defining a strategy interface and a set of strategies implementing this interface. At runtime, which strategy to execute can be dynamically chosen based on certain conditions.

Template Method

The Template Method Pattern defines an algorithm skeleton in one method and defers certain steps to subclasses for implementation. The Template Method Pattern allows subclasses to redefine certain steps of the algorithm without changing its overall structure.

For example, Java’s AbstractList.addAll method is a template method:

1
2
3
4
5
6
7
8
9
10
11
12
13

public boolean addAll(int index, Collection<? extends E> c) {
    rangeCheckForAdd(index);
    boolean modified = false;
    for (E e : c) {
        add(index++, e);
        modified = true;
    }
    return modified;
}

public void add(int index, E element) {
    throw new UnsupportedOperationException();
}

Observer

Define a one-to-many dependency between objects. When one object’s state changes, all dependent objects are automatically notified. This is also known as the publish-subscribe pattern.

The Observer Pattern is a relatively abstract pattern. Depending on different application scenarios and requirements, there are completely different implementation approaches. But the underlying idea is easy to understand, so I won’t include code here. Our use of message queues (MQ) is essentially using the Observer Pattern.

Chain of Responsibility

Decouple the sending and receiving of requests, allowing multiple receivers the opportunity to handle a request. String these receivers into a chain and pass the request along the chain until a receiver on the chain can handle it.

In practice, a variant of the Chain of Responsibility Pattern is also widely used — where every receiver on the chain processes the request.

For example, we have a data assembly logic that’s an application of the Chain of Responsibility variant:

1
2
3
4
5
6
7

private boolean fillUserDataCollection(Map<Integer, UserDataCollection> userExpandMap, Map<Integer, UserDataCollection> originUserExpandMap) {
        return !fillingServices
                .stream()
                .map(filler -> filler.fillUserDataWithFallback(userExpandMap, originUserExpandMap))
                .collect(toSet())
                .contains(false);
    }

UserDataCollection contains many fields, and each fillingService is responsible for supplementing a portion of the data.

The core idea of the Chain of Responsibility is decomposition — breaking large classes into smaller ones. This is also one of the important means of managing code complexity.

Summary

For writing good code, the most critical thing is to first build a mental awareness of what good code is and have the desire to write good code. On top of that, you need a thorough understanding of object-oriented thinking and various design principles, knowing why good code should adhere to these principles.

In practice, the best way to write good code is continuous refactoring. When business development is still unclear at the beginning, it’s normal that code can’t be designed perfectly. As the business develops, we gradually realize the problems in our code. At this point, we can look through design patterns for ready-made solutions and apply them to our own projects.

Finally, I should mention that much of the content in this article actually comes from the Geek Time column “The Beauty of Design Patterns.” I think this is the best column on design patterns and recommend checking it out.

Source: https://lichuanyang.top/en/posts/58527/

Frequently Asked Questions

Q: What if I learn design patterns but forget them without using them?

This is normal. The key is to use the right learning approach. Don’t try to memorize all 23 patterns’ definitions and structures in one go — that’s guaranteed to fade. The correct path: first understand the underlying ideas of SOLID and other design principles, then when you encounter design problems in real projects, refer back to design patterns for ready-made solutions. Apply them purposefully, refactor continuously. As you use them more, the mapping between scenarios and solutions becomes internalized — no deliberate memorization needed.

Q: Which design pattern is the most commonly used?

Among creational patterns, Factory and Builder are used most — you’ll find them in dependency injection and object construction in nearly any project of moderate complexity. Among structural patterns, Proxy (e.g., AOP, RPC frameworks), Adapter (interface compatibility), and Decorator (e.g., Java IO streams) are the most common. Among behavioral patterns, Strategy, Template Method, and Chain of Responsibility appear most often in real-world business code, since they directly correspond to three high-frequency scenarios: “choose an algorithm based on conditions,” “define a processing skeleton,” and “chain-based request handling.”

Q: Which is more important — design principles or design patterns?

Design principles are more important. Principles are the “why”; patterns are the “how.” SOLID, KISS, DRY, and the Law of Demeter guide you in judging the trade-offs of code quality — they are the common foundation for all good code. Design patterns are merely concrete implementations of these principles in certain typical scenarios. Understand the principles, and you can write good code even without remembering a specific pattern’s implementation. The reverse — memorizing patterns without understanding principles — leads to cargo-cult application.

Q: Why has the Singleton pattern become an anti-pattern?

The main issues: Singletons hide dependencies between classes — calling code depends on Singleton.getInstance() without revealing from the interface that global state is involved. They have poor testability — Singletons can’t be mocked, and global state can cause cross-test interference. They don’t support parameterized constructors — limiting flexibility. That said, this doesn’t mean Singletons are categorically unusable. When you genuinely need a single process-wide instance and complex testing isn’t involved, a Singleton remains the simplest choice. The key is understanding its costs, not blindly believing “Singleton = bad.”

First, let me describe two highly inefficient scenarios:

• Xiao A encounters an unfamiliar error during development, so he carries his laptop over to find the team expert Xiao X. Xiao X then works hard to collect all the necessary information. Meanwhile, Xiao A just stands there watching, occasionally chatting with Xiao X and nearby Xiao Y.
• Xiao C receives a large requirement, quickly skims through it, and starts coding. Halfway through development, he realizes he doesn’t understand a certain point in the product spec and goes to ask the product manager. During the conversation, the product manager remembers other things and starts talking about those too. After a long discussion, Xiao C discovers that most of his previous work was wasted.

While these scenarios are somewhat exaggerated, similar situations truly happen all the time. There are many causes for these phenomena, and we’ll analyze them thoroughly today. Before diving into efficiency issues, we need to first look at what programmers do daily. The activities that have the greatest impact on work efficiency can be mainly divided into three categories:

1. Daily development
2. Communication
3. Troubleshooting and problem-solving

Next, we’ll analyze each of these three areas and discuss how to improve efficiency.

Daily Development

What situations in daily development cause efficiency to drop? Let me list a few:

• Rework behavior like Xiao C mentioned above
• Large amounts of repetitive labor in daily work, such as writing SQL, writing boilerplate code
• A seemingly small change that turns out to require changes here and there; worse still, you don’t even know about “the other changes,” so everything breaks when you make the change
• Lack of personal focus, constantly getting distracted

You can see that most of these are related to code design. Regarding design, it really comes down to two points: first, you must design, and second, you must design well.

This means, first of all, you need to know that there should be a design phase before development. Software engineering is a science — it has steps like high-level design and detailed design for good reason. How detailed should the design be before you start coding? My view is: the more detailed the better. In theory, everything except actually writing the code can be done during the design phase.

As for whether the design is good or not, an important evaluation criterion is how easy it is to make changes later. If a single change in requirements causes modifications in multiple places in the code, that’s a poor design. Improving this requires learning about design patterns and architecture — there are no shortcuts.

Additionally, regarding the focus issue, everyone can certainly use tools like Pomodoro timers. But tools can only ever be supplementary; the most important thing is to improve your own self-discipline on a personal level.

Communication

Now let’s talk about communication. Communication can be proactive or reactive. Proactive communication is when you initiate it — going to ask others questions or discuss things. Reactive communication is when others initiate — coming to ask you questions or discuss matters.

The main problem with proactive communication is that conversations tend to drift off topic. Here are two suggestions: first, before initiating communication, you must clearly think about what you want to achieve through this communication. Don’t go into a conversation with just a vague idea — that kind of communication is the biggest time killer. Second, you can make a communication checklist. Topic drift during conversations is actually a common phenomenon. I don’t think there’s a need to deliberately try to prevent it. The key is to ensure you get results on what you originally wanted to discuss, which requires a checklist you can use to verify.

Reactive communication is something that easily disrupts work rhythm, but in many cases it’s unavoidable — sudden production issues to handle, your manager needing to talk to you, etc. What we can do is reduce unnecessary reactive communication. For example, if you provide a feature, write good documentation and comments so others don’t need to ask you to understand it. Method names, return values, and other conventions should be established within the team — when everyone follows them, there’s no need to ask to understand things.

Another point is to break down tasks well — do several small tasks instead of one big one. This way, when you’re forced to interrupt your work, the cost of “context switching” will be lower.

Troubleshooting and Problem-Solving

This is also a very important area. For some people, the time spent finding and fixing bugs may far exceed the time spent developing new features. For this, our main goal is to reduce the difficulty and time of troubleshooting. We can approach this from several aspects:

1. Try to expose problems as early as possible, for example by writing unit tests. The same problem appearing in a small module versus a complex system — the troubleshooting difficulty is self-evident.

2. You need sufficient information, such as logs and various types of monitoring data. It’s like a physical examination — you first need to see the abnormal indicators before you can do targeted, detailed investigation. Otherwise, with no indicators to go on, you’re just blindly guessing.

3. Then there’s the troubleshooting approach itself. Everyone needs to gradually accumulate experience and establish a relatively fixed routine for troubleshooting problems. For example, my habit is to first scan all available information when I encounter a problem, identify abnormal indicators, use search engines or others to understand the indicators if I can’t read them, think about possible causes based on the indicators, and then verify step by step. An important point here is knowing what kinds of problems to think through yourself versus what kinds of problems should be handled with the help of others or search engines. Xiao A’s approach described above is very undesirable — when you ask others or use search engines, the question should be very specific. If you just casually send over an error message, others first need to spend a lot of time collecting information. A problem that could be solved in one hour, if you insist on asking someone else, could easily end up costing two people an entire afternoon.

   What does “specific” mean? For example, if you encounter a ClassNotFoundException and you have no idea what it means, then searching for it is perfectly fine. But if you already know it means the class couldn’t be found, then searching again is meaningless. Instead, you should think about why this class couldn’t be found and what possible causes there might be.

Besides the above, there’s actually another category: the deliberate slacking off that many people choose in the 996 work culture. On this point, I still suggest that everyone use the time freed up by finishing work quickly to improve themselves — slacking off doesn’t help. Another point is to vote with your feet — actively seek out companies and teams without an overtime culture, and use your efforts to help them develop better.

If you have thoughts, feel free to exchange ideas in the comments section or on the public account (Mobility).

Original article: https://lichuanyang.top/posts/3423/

Source: https://lichuanyang.top/en/posts/3423/

Seeing this title, you might think this is going to be about information science. It’s actually not — I just recently observed some things and felt compelled to share my thoughts. I happened to come across two statements online where I happened to be quite familiar with the facts, so I感慨ed that second-hand information is truly unreliable.

Real-World Cases

The first case involves a primary school called Sanfan Affiliated Primary School in the Desheng district of Xicheng, Beijing. Because students can directly enter Sanfan Middle School — one of the top middle schools in Xicheng — this school has always been extremely popular. This year, due to insufficient classroom space, enrollment was reduced from seven classes to four, so the registration residency requirement was raised to four years. Those who hadn’t registered for four years would be reassigned. Actually, the reassignment wasn’t bad, because among the reassignment schools was Xishi Affiliated Primary School, ranked in the top five in Xicheng with ample school places, as well as high-quality direct-entry schools like Leifeng. Some people even bought Sanfan Affiliated Primary School district housing specifically to be reassigned, because Xishi Affiliated Primary School district housing is actually more expensive.

Then, in a highly upvoted comment on a WeChat official account, I saw the claim that Desheng had raised the registration residency requirement to “two or three years” this year. The commenter even used this as a basis to express intellectual contempt for buying school district housing.

Three Levels of Attenuation

Combining the facts with the final statement, you can roughly guess how the information changed throughout the transmission chain. Sanfan Affiliated Primary School in Desheng gradually became just “Desheng,” the four-year residency requirement became “two or three years of residency before being reassigned,” and then further shortened to “requiring two or three years.” Meanwhile, critical information like the fact that reassignment went to Xishi Affiliated Primary School was lost entirely.

The second case: last week, COVID cases appeared in the Wangjing area, and various communities organized mass nucleic acid testing over the weekend. At one testing site, for some reason, the queue was extremely long. However, most testing sites actually had short queues — at some sites, you even had to wait a while to accumulate a group of 5 or 10 people before testing could proceed.

Then this week, I happened to hear a finance influencer talk about this incident, and the information became “tens of thousands of people in the Wangjing area standing in the cold wind queuing for nucleic acid testing.” Actually, what he said wasn’t wrong, and he had no ill intentions — he just mentioned in passing that the pandemic was getting a bit more serious again. But as you can feel, with just a few details missing, the entire impression changes completely. Reading this simplified version, you’d feel quite sorry for the people in Wangjing, but in reality, most people completed their testing easily and happily. As for the site with the long queue, you could always go to another testing site at any time.

So, looking at these two cases — though both are very small matters — I’ve completely lost trust in second-hand information. During transmission, the loss or slight alteration of key details can make the entire meaning completely different. Furthermore, when some people mention things in passing, they don’t consider it an important matter and won’t verify whether the information is accurate. But the listener’s judgment of the matter’s importance may differ significantly. If you make decisions based on such information, you can imagine the consequences.

How to Combat Attenuation

So, what counts as first-hand information? Here’s a simple list:

• Facts. These are things that actually happened — for example, “the temperature in location X today is Y degrees” is a definitive event, whereas “it’s very cold in location X today” is a subjective judgment.
• Professional books. Books go through proofreading processes and have been tested by countless readers, so their credibility is much higher.
• Public policies. I won’t elaborate on this, as they are the standard.
• Data from reliable sources.

In summary, I hope everyone will seek out first-hand information, avoid second-hand information, and especially pay attention to information credibility when making important decisions.

Source: https://lichuanyang.top/en/posts/54296/

Redis primarily includes the following objects: strings, lists, hashes, sets, sorted sets, HyperLogLog, GEO, etc. This article will mainly cover how these objects are implemented. For their commands, you can refer to the official Redis documentation, which provides detailed explanations for each command.

Before diving into specific objects, let’s first take a look at Redis’s definition for the object itself.

1
2
3
4
5
6
7
8
9

typedef struct redisObject {
    unsigned type:4;
    unsigned encoding:4;
    unsigned lru:LRU_BITS; /* LRU time (relative to global lru_clock) or
                            * LFU data (least significant 8 bits frequency
                            * and most significant 16 bits access time). */
    int refcount;
    void *ptr;
} robj;

In a redisObject, the ptr field contains the actual data content. The storage here leverages the low-level Redis data structures we discussed earlier. As you can see, ptr is referenced within redisObject via a pointer, meaning the redisObject itself and the actual data may not be contiguous in memory.

Using the command OBJECT ENCODING KEY, you can inspect the underlying data structure actually used by a key.

Now, let’s dive into each object type.

Strings

Strings are arguably the most fundamental object type in Redis. When you SET a regular key-value pair, you’re using a string object. String objects use one of three underlying encoding types depending on the situation: int, embstr, or raw. The int encoding is straightforward — when the string value is a number, it is stored as an integer. Both embstr and raw use a Simple Dynamic String (SDS) as the underlying structure; the difference is that with embstr, the redisObject and the SDS data occupy contiguous memory. embstr is used when the string is ≤ 39 bytes (using Redis’s default configuration unless otherwise noted), and raw is used for strings larger than 39 bytes.

Lists

A list in Redis is a doubly linked queue that provides push/pop operations from both the left and right sides, as well as range operations.

The underlying implementation of a list can be either a ziplist or a linked list. In the previous article, we introduced ziplist as an extremely efficient data structure for small datasets. So it’s natural that small lists are implemented with ziplist, while larger lists use linked lists. Redis determines whether a list is “small” based on two criteria: the maximum length of each element in the list (configured by list-max-ziplist-value), and the number of elements in the list (configured by list-max-ziplist-entries).

The practical applications of lists leverage the characteristics of a doubly linked queue. For example, WeChat Moments timeline could be implemented using a Redis list. Sometimes, Redis lists are also used as message queues.

Hashes

What we commonly refer to as a map. Similar to list objects, hash objects choose between two underlying structures based on dataset size: a ziplist or a hash table (based on a dictionary internally).

This leads to a useful tip: when you need to store an object in Redis, you can use a hash rather than serializing it into a string. For a typical object, Redis will store it using a ziplist. This way, you can achieve get/set operations on individual fields with minimal resource overhead.

Sets

The set, corresponding to Redis’s S series commands like SADD, SMEMBERS, etc., is a collection of unique elements. Similarly, Redis sets have two implementation methods: intset and hash table.

Redis sets provide convenient methods for computing intersections, unions, and differences of sets, which can be used in scenarios such as finding common friends.

Sorted Sets

Sorted sets correspond to Redis’s Z series commands like ZADD, ZRANK, etc., with the underlying implementation being either a ziplist or a skip list. Having read the previous article’s introduction to these two data structures, it should be easy to understand how Redis implements sorted sets.

Sorted sets are a very commonly used object type, applicable to scenarios such as leaderboards and Top-K problems.

HyperLogLog

HyperLogLog in Redis is a data structure used for cardinality estimation — counting the number of distinct elements. Its implementation is based on probabilistic algorithms, achieving counting over large data volumes with minimal memory usage through an acceptable margin of error. The specific algorithm is quite elegant, and I’ll dedicate a separate article to it later. Here’s a quick overview of its capability: HyperLogLog can count up to 2^64 distinct values using only 12KB of memory, with a standard error of 0.81%.

Using HyperLogLog in Redis is very simple, with just three commands: PFADD to add elements, PFCOUNT to get the current cardinality, and PFMERGE to merge multiple HyperLogLogs.

GEO

As the name suggests, the GEO object in Redis is used for handling location-related information. Redis’s GEO object provides functions for adding and viewing location information, calculating distances between two locations, and finding elements within a range.

The GEO object is built on top of a sorted set (zset) internally. The core mechanism involves encoding two-dimensional latitude and longitude coordinates into a one-dimensional hash value using the GeoHash algorithm, while ensuring that geographically closer coordinates produce closer hash values. We’ll cover GeoHash in detail in a future article. After GeoHash encoding, the underlying zset functionality is leveraged for range queries and similar operations.

Many internet services today choose Redis for their location-based features, such as “nearby people” or “nearby restaurants.”

Through the above introduction, I hope it helps you better understand and use Redis. As you can see, Redis performs extreme memory optimizations for specific scenarios, and when using Redis, we should consider how to take better advantage of these optimizations.

Source: https://lichuanyang.top/en/posts/25564/

Basic Design Principles

Before discussing design patterns, we must first go over the six fundamental design principles. Simply put, design patterns are a series of best practices distilled from these principles.

Single Responsibility Principle

Open-Closed Principle

Solve problems through extension rather than modifying existing implementations. (When writing code, you can assume changes won’t occur; when changes do arise, abstraction is needed to isolate future similar changes.) As long as modifications don’t break existing unit tests, the Open-Closed Principle is not violated.

Liskov Substitution Principle

Subclasses should be able to substitute for their parent classes.

Dependency Inversion Principle

High-level modules should not depend on low-level modules; both should depend on abstractions rather than implementations.

Interface Segregation Principle

Interfaces should be as small as possible.

Law of Demeter

Decouple classes from each other.

Design Patterns

Factory Method Pattern

Define an interface for creating an object.

Abstract Factory Pattern

Create a family of related or interdependent objects.

Singleton Pattern

A class has only one instance.

Builder Pattern

Build each module of an instance separately.

Prototype Pattern

Copy objects.

Adapter Pattern

Handle interface incompatibility.

Decorator Pattern

Dynamically add additional responsibilities to an object, more flexible than creating subclasses.

Proxy Pattern

Provide a proxy for some objects to restrict access to other objects.

Facade Pattern (facade)

The facade contains some or all methods of several subsystems, combining the subsystems and providing them externally.

Bridge Pattern

Separate the abstraction from the implementation. An abstract class depends on an interface, and both parts can be extended independently.

Composite Pattern

Treat a group of similar objects as a single object.

Flyweight Pattern

Shared objects — database connection pools, thread pools, etc. are all applications of the flyweight pattern.

Strategy Pattern

Dynamically switch between multiple different “strategies” to change an object’s behavior.

Template Method Pattern

The parent class defines the skeleton of the algorithm, with some details implemented by subclasses.

Observer Pattern

For one-to-many dependencies, when an object’s state changes, all objects that depend on it are notified.

Iterator Pattern

Provide objects with an iterator.

Chain of Responsibility Pattern

Multiple objects have the opportunity to handle a request, connected in a chain. The request is passed along the chain until one of the objects handles it.

Command Pattern

Pass client requests into an object, achieving decoupling between the “behavior requester” and the “behavior implementer.”

Memento Pattern

A memento object is an object used to store a snapshot of another object’s internal state.

State Pattern

State machine.

Visitor Pattern

Separate data structures from data operations.

Mediator Pattern

Encapsulate a series of object interactions using a mediator object. The mediator makes objects interact without explicit references to each other.

Interpreter Pattern

Given a language, define a representation of its grammar and define an interpreter that uses this representation to interpret sentences in the language. Typical scenarios include SQL parsing, etc.

Source: https://lichuanyang.top/en/posts/21897/

The Essence of Programming as a Career

I want to discuss this from two perspectives. First, from the work itself — in my view, a programmer’s work falls into three categories: discovering problems, analyzing problems, and solving problems. In fact, the vast majority of jobs in the world can be encompassed within these three types of work. Second, from a capability standpoint, I divide abilities into two kinds: the ability to understand the real world, and the ability to map the real world into programming languages.

Regarding the three types of work, in order from discovering to analyzing to solving problems, they progress from macro-level to detail-level. Day-to-day work primarily involves solving problems — all kinds of miscellaneous tasks are fundamentally about solving problems. Discovering and analyzing problems require deeper thinking, but in terms of work itself, I don’t think there’s any hierarchy among these three types. Being able to consistently provide appropriate solutions to identified problems is a formidable core competitiveness in the workplace. As for the two types of capabilities, people generally focus more on the second one. However, I believe the ability to understand the real world is extremely important for a programmer’s career development. Think about it — when it comes to working on business logic or writing CRUD operations, is there a threshold? I think there is — doing business logic well has a very high threshold. Some people always implement business logic in peculiar ways and then blame product managers for unclear requirements, when the real issue may be a lack of understanding of the real world.

Let me elaborate on each of the three types in order.

Discovering problems isn’t strictly necessary for programmers. Many problems, even if you don’t discover them — or discover them and pretend they don’t exist — the system can still run fine. But gradually accumulating such problems will inevitably lead to unmaintainable code. Let me illustrate with a real experience. I once worked on a project involving passenger flight records. We could obtain data for almost all domestic flights. When storing the data, we used ID document numbers as the primary key — because all booking systems use document numbers as their key. But our product displayed data from the user dimension, and a single user could have multiple documents. This meant that even very simple queries — like retrieving a user’s most recent flight record or the last ten records — required complex processing. And as data volume grew, sharding was needed, but documents belonging to the same user couldn’t fall into the same table, adding another level of logical complexity. At this point, not discovering the problem wouldn’t prevent the business from functioning, but working on this piece of business would be extremely painful. Only after recognizing the problem and making adjustments did I realize how simple this business logic actually was.

So how do we discover problems? This requires the ability to understand the real world, as well as sufficient understanding of fundamental concepts like design patterns. To discover problems, we first need a notion of what the “normal state” should look like — anything deviating from that normal state is a problem. In the earlier example, one “normal state” is that querying a user’s recent flight records should be very simple. Another “normal state” is that our user is a person, not a document. When we noticed the deviation from the normal state, we discovered the problem. Some other examples: writing unit tests should be straightforward — when you find unit tests are hard to write, you need to consider whether the code design has issues. A MySQL batch request should take milliseconds, and data volume shouldn’t significantly affect duration — when you notice the relationship between data volume and latency becoming a steep linear function, consider whether you’re actually performing a batch operation (see https://lichuanyang.top/posts/63688).

Analyzing problems is essentially decomposing a manifesting issue into one or more “metrics” that have gone wrong. These “metrics” can be actual measurements like memory or CPU, or they can be basic conceptual understandings and design approaches. In the earlier example, the problem we discovered was that very simple requirements were difficult to implement. The “metric” that eventually turned out to be the root cause was the choice of primary key for storing records.

For analyzing problems, you should make reasonable conjectures and then use tools to verify or disprove your guesses. Once I discovered data wasn’t being saved to the database. I could form several reasonable hypotheses: the database itself had an issue; the connection to the database was problematic; the transaction wasn’t committed normally; logic errors prevented data from being generated; and so on. Then I checked the logs for errors, looked at MySQL monitoring for uncommitted transactions. Step by step, the problem was pinpointed.

Depth vs Breadth

Solving problems is what most programmers spend the most time doing. For a programmer, solving problems means representing real-world concepts in a computer. The two most critical aspects, in my understanding, are: first, decomposing tasks — breaking a large task into independent subtasks; and second, the ability to quickly understand and use new tools — essentially, continuous learning. Many people complain that there’s too much to learn as a programmer. But you must pay attention to what’s worth spending significant time on and what you only need to skim through. I’ve noticed many people are enthusiastic about learning tools, enjoying writing various versions of Hello World — in my view, this is ineffective learning. Programming learning requires both depth and breadth. Depth means thoroughly researching certain technologies — after studying some, you’ll find that various technologies, which may seem unrelated on the surface (like databases and message queues), actually share many similar concepts and ideas. Once you’ve studied some in depth, you can build your own knowledge system, and then learning new things becomes much faster.

Because every new technology emerges from the urgent need to solve practical problems. For example, why was ClickHouse created? Because row-storage databases like MySQL aren’t suitable for data analysis, and there was no columnar storage database with both good performance and stability. Similarly, Jedis connecting directly to Redis Server doesn’t perform well in multi-threaded environments, which is why Lettuce was created. Once you understand what problem needs to be solved, try thinking about what solution you would propose. Then look at the actual solution. If your approach is similar — congratulations, your understanding of technology has reached a new level. If it’s different — congratulations again, you’ve learned a new way of thinking. These insights can be applied whenever you work on business logic in the future.

My Career Philosophy

Let’s keep growing together.

Source: https://lichuanyang.top/en/posts/27398/

Why Use Bitwise Operations for Case Conversion?

Your first reaction might be: doesn’t Java already have Character.toUpperCase() and Character.toLowerCase()? Why reinvent the wheel?

Indeed, 99% of the time you should use the standard library. But understanding this trick has several benefits:

1. Common interview topic: Many algorithm interviews test this kind of bitwise thinking
2. Understanding low-level computing: Appreciate the elegance of ASCII code design
3. Embedded/low-level development: In environments without standard libraries, this is how it’s done
4. Code brevity: ch ^= 32 is far more elegant than Character.isUpperCase(ch) ? Character.toLowerCase(ch) : ch

The Elegant Design of ASCII

To understand this trick, first examine how English letters are encoded in the ASCII table:

Notice the pattern: the ASCII values of uppercase and lowercase letters differ by exactly 32 (0x20). At the binary level, this is the difference in the 6th bit (counting from the right, starting at 0):

• Uppercase letters have the 6th bit as 0 (e.g., A: 0100 0001)
• Lowercase letters have the 6th bit as 1 (e.g., a: 0110 0001)

This is the beauty of ASCII’s design — case conversion requires flipping just a single bit!

Three Bitwise Approaches

1. XOR (Toggle): Switch Between Cases

1

ch ^= 32;  // equivalent to ch ^= 0x20

The ^ (XOR) rule is “same gives 0, different gives 1”. XOR-ing with 32 (where the 6th bit is 1) flips only that bit: uppercase becomes lowercase, lowercase becomes uppercase.

2. Force to Uppercase

1

ch &= ~32;  // equivalent to ch &= 0xDF

~32 (bitwise NOT) makes the 6th bit 0 while leaving all other bits as 1. The & operation clears the 6th bit, so regardless of input case, the output is always uppercase.

3. Force to Lowercase

1

ch |= 32;  // equivalent to ch |= 0x20

The | operation sets the 6th bit to 1. Regardless of input case, the output is always lowercase.

Edge Cases

The above operations only work for English letters. They are meaningless for non-letter characters, so validate first:

1
2
3

if (ch >= 'A' && ch <= 'Z' || ch >= 'a' && ch <= 'z') {
    ch ^= 32;
}

What About Real-World Performance?

On modern JVMs, Character.toUpperCase() has been optimized as an intrinsic (JVM-level native optimization), so there is virtually no performance difference. The takeaway: understand this trick, but prefer the standard library in daily development. Bitwise case conversion is most useful for algorithm problems, embedded scenarios, and understanding open-source code that employs this technique.

In a previous article, I introduced the basic syntax for using ActiveMQ on the client side.

It is worth noting the consumer side:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

    public void consume() throws JMSException {
        ActiveMQConnectionFactory cf = new ActiveMQConnectionFactory();
        cf.setBrokerURL("tcp://localhost:61616");
        Connection connection = cf.createConnection();
        connection.start();
        Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
        Queue queue = session.createQueue("TEST");
        MessageConsumer consumer = session.createConsumer(queue);

//        Message message = consumer.receive();
//        TextMessage textMessage = (TextMessage) message;
//        System.out.println(textMessage.getText());

        consumer.setMessageListener(message -> {
            try {
                System.out.println("consumer1: " + ((TextMessage) message).getText());
            } catch (JMSException e) {
                e.printStackTrace();
            }
        });

        Session session2 = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
        Queue queue2 = session2.createQueue("TEST");
        MessageConsumer consumer2 = session2.createConsumer(queue2);
        consumer2.setMessageListener(message -> {
            try {
                System.out.println("consumer2: " + ((TextMessage) message).getText());
            } catch (JMSException e) {
                e.printStackTrace();
            }
        });

        Queue queue3 = session2.createQueue("TEST");
        MessageConsumer consumer3 = session2.createConsumer(queue3);
        consumer3.setMessageListener(message -> {
            try {
                System.out.println("consumer3: " + ((TextMessage) message).getText());
            } catch (JMSException e) {
                e.printStackTrace();
            }
        });

        try {
            TimeUnit.MINUTES.sleep(100);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }

    }

Multi-threaded Consumption Analysis

As you can see, there are actually three ways to achieve multi-threaded consumption of the same queue: defining multiple connections, sessions, and consumers. At the application level, these three approaches produce the same effect — they all generate multiple consumers to process messages in the queue in parallel. However, in terms of performance, there are noticeable differences among these three approaches. Let’s elaborate below.

First, defining multiple consumers is only a form of pseudo-parallelism and does not achieve true concurrent consumption. The reason is that in the JMS protocol, a session can only be used by one thread at a time. So when multiple consumers share the same session, they simply take turns using that one session.

Multiple sessions and multiple connections, on the other hand, are truly concurrent operations. A connection corresponds to a physical TCP connection. One connection can create multiple sessions, and these sessions share the same TCP connection. The difference between multiple sessions and multiple connections is simply whether you open one or multiple TCP connections. The impact is easy to understand: multiple TCP connections can handle greater network traffic, but of course they also bring some overhead in establishing and maintaining connections.

Summary and Recommendations

In summary, in most cases, you can define multiple sessions to achieve parallel consumption in ActiveMQ. When traffic is high, you can consider opening multiple connections. As for multiple consumers, I have not yet found a scenario where that would be useful.

Source: https://lichuanyang.top/en/posts/20459/

Problem Description

For a billing system, concurrency issues are actually divided into two categories. The first is high application concurrency—essentially a large number of users and high access volume. This type of problem is no different from general high concurrency problems and can be solved using distributed techniques. The second category is user concurrency issues that generally cannot be solved by distributed methods alone, which is what this article will focus on.

This type of problem stems from high-frequency access to certain accounts. A large number of concurrent accesses cause bottlenecks to first appear at certain database records. Many operations end up waiting because they cannot acquire row locks on the database, and these waiting operations consume other resources, eventually leading to system unavailability.

The following sections introduce some common approaches to handle this type of problem.

Not Setting a Balance Field

Since a stable billing system must record billing transaction details, it’s entirely possible to not have a balance field and instead calculate the balance based on transaction details.

However, this method is not universal. For example, in an advertising billing system where the frequency is very high and each transaction amount is very small, trying to calculate the balance by summing up transactions is clearly impractical.

Merging and Splitting

These are two different approaches, but since they share some similarities—both aim to reduce access pressure on individual database records—I’ll discuss them together.

Merging involves consolidating multiple requests for a single account before writing to the database, effectively reducing the pressure by several times.

Splitting involves breaking a main account into several sub-accounts and distributing requests across them, reducing the pressure on any single account. Then, other techniques are used to merge the sub-account data into the main account data for returning to the user.

Reducing Row Lock Hold Time

This is a code-level optimization. As mentioned earlier, the reason high-frequency accounts cause system performance issues is the contention for row locks. So if we can reduce the time each request holds a row lock, system performance will improve significantly.

First, try to speed up the execution between acquiring the row lock and committing the transaction. Move unnecessary operations, especially time-consuming ones, to be executed elsewhere—either before acquiring the row lock or outside the transaction.

Then, try to avoid acquiring row locks using:

1

select ... for update

Instead, use the following approach:

1

update xxx set amount=amount-1 where id=x and amount>=1

If the business logic allows negative balances, you can omit the amount validation in the WHERE clause. Otherwise, when the balance would go negative, you should skip the database update and return an exception in the application code.

Rate Limiting

Since high-frequency accounts only cause system performance issues when a single account’s concurrency reaches a certain level, we can forcibly control this concurrency level to keep it within an acceptable range for the system.

Caching

Caching is also a common approach for handling high-frequency accounts. Operations are performed on the balance in the cache, and the data is synchronized to the database periodically.

The above methods each have their applicable conditions and limitations. For example, merging and rate limiting both result in delayed deduction. During this delay period, the account balance might already be exhausted. So for business scenarios where the balance strictly cannot go negative and records cannot be discarded, these approaches may not be suitable.

In summary, the most important thing is to choose the appropriate solution based on the business scenario.

Source: https://lichuanyang.top/en/posts/56940/

I encountered an interesting problem today while scanning code with FindBugs — it’s about the ternary operator. Let me record it here.

It’s about code like this:

1
2

boolean b = true; 
Long a = b ? 0l : Long.valueOf(2); 

FindBugs gave the warning: “Boxed value is unboxed and then immediately reboxed” — meaning a boxed object is unboxed and then immediately reboxed. This problem is actually quite common. I didn’t pay much attention to it at first, and just habitually changed Long.valueOf to Long.parseLong, which did eliminate the warning. But later I realized something was wrong: valueOf returns a Long type while parseLong returns a long type, and what we need is actually the Long type. So why does using valueOf cause a problem while parseLong doesn’t?

If you think about it, it’s actually straightforward. The issue lies in the other branch of the ternary operator — because the other branch returns an unboxed 0, the return type of this ternary operator becomes long, so the original Long type must go through an unboxing operation before being returned. To optimize this, just make sure both branches have consistent return types.

Then I looked up the related details and got a clearer understanding.

Autoboxing/Unboxing

Since JDK 1.5, Java has introduced autoboxing and unboxing, eliminating the need for explicit type conversions and improving our development efficiency. For example:

1
2
3
4
5
6
7

Double dWrap1 = 10d; 

double d1 = dWrap1; 

double d2 = d1 + dWrap1; 

DoubledWarp2 = d2 + dWrap1; 

This code can run normally.

Another thing to note is that in an expression involving type conversion — such as the ternary operator mentioned earlier — the compiler will prioritize primitive types, meaning it will first unbox already boxed objects.

Our original problem was just a minor performance overhead:

1
2

Long B = null; 
Long A = (2>1)?B:0l; 

But code like this is actually buggy. It looks like we’re just assigning a null to a Long-typed A, but during this process, an unboxing to long will occur, which will definitely cause a NullPointerException.

Therefore, in our daily development, we should try to avoid unnecessary boxing/unboxing and type conversions. Not only for performance reasons, but also to avoid some strange issues.

Source: https://lichuanyang.top/en/posts/53072/

AsyncLoopThread is a custom loop task execution utility in JStorm. The implementation is not complex, and on its own may not warrant a dedicated article. However, it is so widely used in JStorm — for features such as supervisor/nimbus heartbeat, fetching new topologies, updating worker status, and many more — that it’s worth introducing here, as it will also help with reading other parts of the codebase.

Usage

AsyncLoopThread can actually be extracted and used in your own projects. It’s quite convenient to use. You can refer to https://github.com/lcy362/Scenes/blob/4d8ec4ff166060cf5d491c33a96f5c86e8389333/src/main/java/com/mallow/jstormcode/AsyncLoop.java

You only need to first define a RunnableCallback class. RunnableCallback is a thread class wrapped by JStorm that enhances the Runnable interface with callback support and the ability to proactively shut down.

1
2
3
4
5
6
7
8
9
10
11
12

public class TestThread extends RunnableCallback {
    @Override
    public void run() {
        System.out.println("thread runs " + new Date());
    }

    @Override
    public Object getResult() {
        return 10;
    }

}

Here we only implement two methods: run and getResult. The run method is the thread execution method, same as a normal thread. getResult returns the task execution interval time, in seconds.

After that, you just need to pass this TestThread as a parameter to AsyncLoopThread, and it will implement the loop execution of TestThread’s run method.

1
2
3
4
5
6
7
8
9
10

public static void main(String args[]) {
        AsyncLoopThread loop = new AsyncLoopThread(new TestThread());

        try {
            TimeUnit.MINUTES.sleep(5);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

Design

Next, let’s look at the specific implementation of AsyncLoopThread.

The core code is the init method:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

private void init(RunnableCallback afn, boolean daemon, RunnableCallback kill_fn, int priority, boolean start) {
    if (kill_fn == null) {
        kill_fn = new AsyncLoopDefaultKill();
    }

    Runnable runnable = new AsyncLoopRunnable(afn, kill_fn);
    thread = new Thread(runnable);
    String threadName = afn.getThreadName();
    if (threadName == null) {
        threadName = afn.getClass().getSimpleName();
    }
    thread.setName(threadName);
    thread.setDaemon(daemon);
    thread.setPriority(priority);
    thread.setUncaughtExceptionHandler(new UncaughtExceptionHandler() {
        @Override
        public void uncaughtException(Thread t, Throwable e) {
            LOG.error("UncaughtException", e);
            JStormUtils.halt_process(1, "UncaughtException");
        }
    });

    this.afn = afn;

    if (start) {
        thread.start();
    }
}