---

From left:
Tetsuya Kawase — Technology Division, Product Development Group, Vista Department
Nozomi Ito — MagicPod CEO

---

Kawase: I joined Hacobu in August 2023. I am responsible for QA on the dispatch order and management service "MOVO Vista." Hacobu does not have a centralized quality assurance department that spans the entire organization. Instead, we have built a "Product QA" structure in which a dedicated QA professional is embedded in each product team to develop and execute the optimal test strategy for that product.

With this structure, the consensus-building needed to launch new initiatives is handled entirely within the team, which means QA decisions directly shape the test strategy and quality improvement efforts for the whole team. I find it easier to move forward with genuine buy-in, and the agility it enables is a significant advantage. We have nine full-time QA engineers, each operating independently, and we hold a sharing session roughly every two weeks. We also have active informal groups — similar to guilds — where members with shared interests come together in small groups to work on things collaboratively.

I originally started my career as a software engineer, spending about 14 years at an SES company working on system development for other organizations. In the latter part of that period, I increasingly moved into roles overseeing entire teams and projects, and on smaller engagements I had more and more opportunities to handle scenario testing and integration tests.

After that, I joined another company in a business planning and SI vendor liaison role, and became involved in an app development initiative the company was running at the time. Through that experience, I discovered the interest of engaging not only with the builder's perspective of quality — "making sure the system works" — but also with quality from a user's point of view: "what does it take for customers to be genuinely satisfied?" That sparked my interest in the field, and after gaining experience at a third-party verification firm, I joined Hacobu.

---

Reasons and Background for Selecting MagicPod

Kawase: Product QA is one of Hacobu's strengths, but it also carries the risk of over-reliance on individual knowledge. When I joined, the company was in a phase that prioritized the speed of agile development, and regression testing had not been properly established — the prevailing understanding was that manual testing would cover what it could. The judgment of which tests to run depended on the discretion of each individual, and the situation was such that incidents like "there's a regression somewhere but we can't identify the cause" could easily arise.

Ito: Was it you who drove the adoption of automated testing and the selection of a specific tool?

Kawase: It had already been under consideration before I joined. It seemed to be a concern at the leadership level as well, and when MagicPod was introduced, our CEO responded positively. After I joined, the evaluation came down to a choice between a recording-type tool and MagicPod. In that process, we assessed MagicPod favorably for its straightforward creation flow and high maintainability. Personally, I also appreciated that Shared Steps were available.

Incidentally, I first came across MagicPod at my previous job at a third-party verification firm, when a client company was exploring automation of their regression testing. Another person was leading that effort so my involvement was limited to trying it out, but they ultimately adopted it there as well.

Ito: Many companies appreciate the ability to run tests without limits — was that not a significant factor for you?

Kawase: At the time, I don't think I had a concrete enough picture to fully appreciate it. That said, hearing you mention it now, being able to run tests every day without hesitation is genuinely a major benefit.

Ito: Thank you! Were overseas testing tools or Selenium also among the candidates?

Kawase: In the earlier rounds of evaluation, yes, those were on the list. However, recording-type tools — which are common among overseas products — tend to depend heavily on how each individual builds tests, making it difficult to maintain consistent quality. Coding-based tools, on the other hand, raised concerns about the onboarding learning curve, since programming experience varied across team members, and so they were ruled out. Ultimately, we selected MagicPod for its ability to support structured test creation and its high maintainability.

Ito: With product teams operating separately under a Product QA structure, I imagine it can be difficult to drive adoption across the organization. How did you approach that?

Kawase: Initially there was another person driving the effort alongside me, and that person first introduced MagicPod in the truck appointment scheduling service "MOVO Berth." Through that process, we established a full set of review mechanisms, review criteria, and test creation guidelines, and when rolling out to other products, we applied those frameworks while continuing to refine them.

I believe that in test automation, what matters more than the tool itself is designing the structure through which quality is assured. It is often misunderstood that "because it's no-code, no design is necessary" — but my view is that "no-code ≠ no design."

In practice, when teams jump straight into implementation without sufficiently thinking through the test structure, data handling, and rule design — the "architecture" of automation — changes become difficult to make later, and the result is often what I would call a hollowing-out of the automation: failed tests get left unresolved, and the whole effort falls into disuse.

At the same time, the mechanics of how to use the tool itself can be picked up along the way through hands-on practice. That is precisely why our team prioritized getting the hard-to-change, low-leverage-once-set elements — test structure, data design, and operational rules — sorted out before we began implementation.

---

Foundation Design in the Early Stages of MagicPod Implementation

Ito: What specifically did that preparation involve?

Kawase: In software development, coding standards and variable naming conventions are typically in place to ensure a baseline of quality. Drawing on my own background as an engineer, I applied a similar approach to test automation and organized the groundwork into four broad layers.

The first was structural design, with post-implementation maintainability and ease of implementation in mind. By clarifying which MagicPod features map to which traditional test structure concepts, the team was able to develop shared understanding and move forward with implementation on a common foundation.

---

Next, we established guidelines and naming conventions to enable safe Data Driven Testing. Naming rules are differentiated by variable type — for example, shared variables use uppercase snake_case, while variables within test cases use camelCase.

---

The third layer was operational design. Establishing structure, guidelines, and conventions alone is not enough to ensure they are actually followed, so we documented the implementation flow for test cases, the review flow, and the process from investigating a test failure through to resolution.

Finally, we prepared a "usage guide." In the early days after implementation, test case creation was a process of trial and error, so we set up a page to accumulate observations and anti-patterns from implementation, allowing the team to share good practices and patterns to avoid. Through this preparation, I believe we were able to build a foundation where operations would not become hollow over time and where adjustments would remain manageable.

Ito: That is impressive. It is rare for an organization to have this level of preparation in place before introducing a tool. I felt that the "no-code ≠ no design" philosophy is genuinely reflected in how you operate.

Kawase: Thank you. That said, maintaining this operational foundation over the long term also required keeping the team motivated in the early stages. What proved helpful there was MagicPod's analytics feature. In particular, we set stabilizing the "health score" — which measures the health of operations — in the 90s as our near-term goal. Because the evaluation criteria are broken down into incremental stages with the importance and priority of each item displayed, it was easy to set short-term milestones, and I felt that contributed to sustaining motivation.

---

How MagicPod Is Being Used

Kawase: Currently, we have automated testing for MOVO Berth, MOVO Vista, and a third — including a smartphone app — bringing the total to three products covered by MagicPod. Accounts have been issued to developers as well, bringing the total to around 50 users. Depending on the team, in the product I am involved in, MagicPod is used as part of the release approval process, and there are cases where developers independently run MagicPod outside the regular release schedule to make their own release judgments.

Scheduled Execution kicks off at 7:30 in the morning and completes in about 45 minutes, with results delivered to a dedicated Slack channel that only receives execution results. In general, when a Batch run test fails, QA investigates — but members who are engaged will proactively look into the error on their own. They first determine whether the issue is transient, and if it turns out to be a defect on the frontend or backend, a fix is applied and the release approval run is executed again.

There was actually a point in the past where production releases were going out before the daily test results had been reviewed, and we had a problem with testing and releases being disconnected. So we established a clear release standard: "all morning tests must have passed." For runs outside that window as well, we use a workflow automation tool — a Slack command triggers a Batch run automatically.

Ito: It sounds like your development engineers are also engaging with MagicPod. Have there been any positive responses since the rollout?

Kawase: We have received positive feedback about the fact that defects that previously slipped through testing are now being caught before release. There is a shared understanding across the entire team that proper verification happens before a release goes out, and I feel that the resulting sense of assurance is a visible effect. Beyond that, incorporating MagicPod into the release approval process has led to engineers proactively asking "Did you run MagicPod?" — and a culture has emerged where the team locks down release content by the day before to make it in time for the morning tests. That improvement in quality awareness across the whole organization has been one of the most significant outcomes.

Ito: It is wonderful to see such a healthy culture taking root across the development team. For that kind of stable daily operation to be sustainable, the maintainability of the test cases themselves also becomes important. Do you have any criteria for granularity when creating Shared Steps, or any tips for improving maintainability?

Kawase: As a baseline, we always implement API calls as Shared Steps — handling everything from the call itself through to result retrieval and return as a single unit. Beyond that, our policy is to actively consolidate anything that is used frequently or corresponds to shared UI components on the frontend.

Another practice we use to improve maintainability is an automated review system leveraging AI. Specifically, we integrate Cursor with MagicPod's MCP. We preload all of our coding rules and variable naming conventions into Cursor's configuration and created a custom command from there.

When you run this command and specify a test case number, the AI automatically performs a review against the conventions and feeds back the points that need to be corrected. In practice, each team member runs this automated review in their local environment, writes the output to Notion, and then uses that as the basis for requesting a final confirmation review from other team members. Once there are no issues, the test case moves into live operation.

Beyond reviews, we also run processing via Cursor through the MCP server to re-execute only the tests that failed in a Batch run, and to fetch data via the API and generate graphs showing trends in execution time. Aggregation and analysis tasks that would take more effort through the GUI can be executed simply by entering a natural language prompt, and I feel that has significantly expanded the freedom we have in managing test automation operations.

Ito: It sounds like quite a sophisticated setup — automated AI-powered reviews, aggregation via MCP, and more. Going forward, how are you thinking about rolling these practices out to other team members and teams within the company?

Kawase: Our organizational culture places a very strong emphasis on individual autonomy and team-level optimization. So rather than managing things top-down, the challenge going forward is how to expand these practices while leveraging the initiative of people on the ground. That said, one of Hacobu's strengths is a culture where proposing to try a new tool gets a "let's go for it" response from the company, and where lively discussion happens across team boundaries on Slack and elsewhere. I think the path forward is to leverage that environment to spread adoption organically.

---

Toward a QA Organization That Keeps Pace with the Speed of AI-Driven Development

Ito: What challenges are you hoping to address going forward with the help of AI?

Kawase: Proficiency with AI varies across teams and individuals. Some members struggle to effectively translate AI suggestions into test cases, while others find themselves in trial-and-error mode when unexpected side effects arise during fixes. There is also a significant challenge around not being fully aware in advance of which MagicPod test cases will be affected by a given change — even when those tests are part of the release approval criteria, the impact sometimes only becomes clear after the fact. That invisible blast radius is something I want AI to help us detect.

Ito: You mean identifying the scope of impact from a change. One approach that has emerged recently is to pass a GitHub pull request URL to an AI editor like Cursor via MagicPod MCP and instruct it to "list the tests that are likely to be affected by this change" — which allows for a reasonable degree of pre-run impact assessment without actually executing the tests.

Kawase: That sounds very useful — I'll look into it. Our development environment is quite complex, with dev and staging environments running alongside topic environments for each development unit. When resources allow, we can do comprehensive checks in the topic environment, but when things get busy, there are inevitably cases where verification ends up being limited to the dev environment.

Inheriting tests created by someone else is particularly challenging — it is not easy to read back the design thinking behind them. As a result, the tendency is to apply local fixes only to the step where an error occurred, without considering the overall structure, and that is one of our significant operational challenges.

Ito: I see — the more complex the environment, the more important it becomes to triage before running tests. Being able to reduce unnecessary executions through early detection is a meaningful benefit. And beyond just fixing the error location, supporting developers in making corrections that account for the broader context — in a way that is simpler and more accurate — seems like it will become increasingly important going forward.

Kawase: Exactly. On that point of "support for making the fix," I feel that MagicPod Autopilot would become even more powerful if it were extended to support Shared Steps. Even now, I have confirmed that when data patterns are in use, Autopilot appropriately retrieves the relevant variables — and that seems like something we can put to use. For things like calculation logic that I don't work with often, it feels easy to delegate the finer details to it.

Ito: We talked about Cursor and MCP for impact assessment earlier, but there is a step beyond that worth mentioning as well. Depending on how it fits your operational flow — are you currently using the approach of reviewing code in an AI editor and then applying those review results directly to test fixes via MCP?

Kawase: We are not doing that yet.

Ito: Actually, it has recently become possible to call Autopilot from MCP, so that by instructing the editor to "apply these review results to the tests as well," new test creation and modifications can be carried out automatically. At this point in the current specifications, changes are applied directly to the main branch rather than a test branch, so there is one extra step needed — checking the execution history to confirm everything looks correct.

Kawase: For newly created tests, having changes applied directly to the main branch is completely fine. That sounds very useful — I will definitely try it. Thank you.

Recently, AI-powered code generation and similar tools have dramatically accelerated the speed of feature development, and there is a shared sense of urgency within the company that if QA remains reliant on traditional manual testing, it will become a bottleneck for the entire release process.

Ito: How to keep QA pace with development speed as AI continues to accelerate it is a major theme for the industry as a whole. What will be needed going forward is not disposable tests written purely for speed, but tests that are resilient to change and built for stable long-term operation. MagicPod will keep evolving to support that.

---

Closing

Kawase: In our initial meetings after implementation, you shared common failure patterns and operational tips with us upfront — and that enabled us to design a structure with long-term operation in mind from the very beginning. Looking back on these past two years, I am genuinely grateful that we got that initial foundation right.

I think most engineers today understand the value of automating regression testing. But the true value, I would argue, lies in being able to achieve that "without depending on any one individual's technical ability — with consistent quality regardless of who touches it." My honest assessment is that adopting MagicPod was the right call — as a tool for embedding an automation culture across the organization and eliminating over-reliance on individuals.

Hacobu Co., Ltd.

• Corporate site: https://hacobu.jp/
• Careers: https://career.hacobu.jp/
• Tech blog: https://zenn.dev/p/hacobu
• Tech entrance book: https://hacobu.notion.site/entrance-book-engineers

---

From left:
Takahiro Yamauchi, Product & Service Unit
Mitsuhiro Oishi, Product & Service Unit, General Manager
Nozomi Ito, MagicPod CEO

---

Mitsuhiro Oishi (hereafter Mitsuhiro): I joined the company in 2008, when the development team had about 10 engineers. In 2012, we launched PORTERS — a matching system specialized for staffing and recruitment, and have continued updating it ever since.

PORTERS is a SaaS platform that centrally supports the core operations of staffing businesses, from job order management and candidate management to hiring progress tracking and revenue management. Because we continuously expand our feature set to accommodate the industry's complex workflows, figuring out how to build an organization that balances development speed with quality has been my overarching challenge.

Testing was originally handled by the development engineers themselves. Around 2011, our CTO at the time joined and established a QA team, and as the service grew, our quality assurance structure gradually took shape. For test automation specifically, that has been Takahiro's domain.

Takahiro Yamauchi (hereafter Takahiro): I joined PORTERS as a contractor about two years ago. I've been involved in testing for around 20 years — as a QA chief, team lead, and sometimes as a tester. On top of that I had been driving automation at my previous company as well. However, the tool we introduced there would sometimes fail to run scenarios we had created, and other team members were struggling with the same issues, so I had developed the impression that automated testing was something that just didn't work reliably.

---

Challenges Before Implementing MagicPod

Takahiro: Actually, before joining PORTERS, I was involved in PORTERS' testing as a member of an external company. I wasn't directly testing the PORTERS service itself, but even then, looking at the sheer volume of features, I thought, "the people testing all of this must have it rough." When I actually joined as a contractor and got inside, it was exactly as I had imagined. (laughs)

Mitsuhiro: Our QA operations are currently handled by VNEXT, a Vietnamese offshore company. They manage everything from test case creation to manual test execution. But as PORTERS' feature set kept growing, the time required for testing became a serious problem at a certain point.

The moment we felt the most pressure was during middleware version upgrades. Version upgrades are essential for maintaining strong security, but when a full test cycle takes three months, it means no new releases can go out during that entire period. Our biggest concern was development coming to a halt when we wanted to be delivering more value to our users — we had reached a point where manual testing simply wasn't sustainable anymore.

On top of that, there were areas where a programming error could lead to information leaks — such as email sending and external data integrations. We had a strong desire to automate testing for those areas to bring regression defects as close to zero as possible.

Three or four years ago, our engineers had actually tried building automated tests using Selenium. With features expanding as the service grew and manual testing falling behind, everyone pitched in to write code and we made it through that period. But once the engineers returned to development work, there was no bandwidth left for maintenance, and the automation ultimately never became a sustainable operation.

---

---

Why and How MagicPod Was Selected

Nozomi Ito (hereafter Nozomi): With those challenges in mind, how did you go about selecting a tool?

Mitsuhiro: We compared about three options, and the deciding factors were the number of test executions and ease of implementation and operation.

On the features side, MagicPod has no limit on the number of test executions. Because we release frequently to deliver value to our customers as quickly as possible, and because we need to run tests with every security update, we determined that any cap on executions would make it impossible to keep up.

On the operations side, our experience with Selenium had taught us that we needed to avoid tools so complex that only a handful of people could use them. Knowing that MagicPod could realistically take root on the ground and be continuously maintained was another major reason for choosing it.

We completed the selection in about a month and have been using it since November 2023. We started by having the VNEXT team study MagicPod and building test cases for a subset of features such as email-related functionality — however, at that initial stage we weren't able to get it running smoothly. So we went looking for an external partner to help formalize our automated testing operations, reached out to about three companies, and ultimately brought in Verisurv.

Verisurv has deep expertise in MagicPod, and what really made the difference was that they said they would thoroughly share their knowledge with the VNEXT team as well.

Takahiro: After Verisurv came on board, I also started engaging with MagicPod in earnest. Watching their approach to knowledge sharing and maintenance, I found it completely different from the tools I had used before. It was easy to build in, easy to read, easy to verify. I thought, "this is really good."

---

---

Using MagicPod

Mitsuhiro: Verisurv supported us for about six months, during which we built test cases starting from the most critical features. That structure has now found its footing, and the VNEXT team is able to operate autonomously. Takahiro manages things from the Japan side, with a VNEXT team of 13 people on manual testing and 4 on automated testing.

Because the manual and automated teams both sit within VNEXT, we're able to create a strong feedback loop — defects caught by automated tests get fed back into the manual test cases, and conversely, bugs found during manual testing of new features get incorporated into the automated test scenarios. That mutual integration has been hugely valuable.

Takahiro: Execution runs mainly overnight and during daytime hours on weekends, which are times when team members aren't working — running 2 instances in parallel. The number of test cases has grown considerably, so even with parallel execution it takes about three days to complete a full cycle. The QA team checks the results first, and if there are any new errors or scenarios that stopped, I get notified. I then review the results on my end as a double-check to make sure nothing was missed.

One thing that's particularly helpful from an operational standpoint is how much easier the test results are to read. Screenshots are captured at the point of failure, so you can immediately see what was being done, where, and what happened. I also find it very useful that when a test encounters an error, you can choose whether to stop there or continue running — that flexibility matters a lot in practice.

For example, if execution stops because of a minor error like a slight text difference, you have no way of knowing whether everything after that point is fine or not. Being able to continue running through an error and check all subsequent behavior in one go is a significant operational advantage.

Mitsuhiro: When we ran the numbers, we had roughly 30,000 test cases in total. Running all of them as manual tests cost approximately 6 million yen in labor per cycle. By incorporating automated testing, we've been able to bring that down to approximately 1 million yen — a reduction of roughly 5 million yen per full test run.

On top of that, the full test period has been cut from three months to approximately two weeks, which means development no longer has to stop during testing — and that impact is enormous. The system has also been reliably catching defects, detecting multiple regression issues, which has been a huge help.

Takahiro: A little while after I joined, I asked the manual testing team, "How long would it take to do a full test right now?" They said, "Six months." Now, with the exception of some areas like admin screens, we've reached a state where we can cover nearly all the features a standard user would interact with.

The execution environment is also flexible — simply changing the base URL lets us switch between production and staging, so we can check for regression defects in the customer-facing environment or verify that there are no regressions outside of new features in the development environment. Automated testing has also become well understood within the development team. We now regularly get messages asking, "How soon could you complete a full cycle if we requested it now?"

Mitsuhiro: There's been a change in terms of knowledge concentration as well. PORTERS has accumulated over a decade of features, and even current development team members regularly encounter things like, "I didn't know this feature existed." Automated testing covers those areas that tend to get overlooked, and whenever a regression occurs, we add a test case so the test suite grows over time. Having a system where, even as team members change, things like "this area slipped through the cracks" become less likely to happen. I think that's genuinely valuable.

Nozomi: Are there any practices or approaches you've developed to keep daily operations running smoothly?

Mitsuhiro: Actually, we've intentionally chosen not to use the notification feature. Every day, the VNEXT team members go directly to check the results themselves to see whether any errors have occurred. Automated testing will always have things that don't go perfectly, so rather than relying on notifications, having dedicated people doing daily checks is something we think is essential to keeping the operation running continuously.

Takahiro: There's also a specific approach we've developed for handling the quirks of automated testing. For example, in tests that verify email body content, MagicPod performs strict comparison down to invisible line break differences, which can cause failures. But in practice, there's no visible impact on the customer-facing screen. For defects that only appear in automated testing, rather than forcing changes to the scenario, we've adopted the practice of letting those run as known failures. We believe it's important not to get too caught up in errors that have no customer impact, and to make judgment calls with flexibility.

---

---

From "Automate for Now" to Sustainable Test Operations

Nozomi: You mentioned that there was a period early on after implementing MagicPod where things didn't get off the ground smoothly. What specifically made it difficult?

Mitsuhiro: At first, we tried to directly replace manual tests with automated ones. We were building in granular checks like "did this text change or not?" But what matters in automated testing isn't that — it's whether the features are functioning correctly.

When you're checking areas where minor text differences don't matter, execution becomes a burden and failures pile up to the point where it stops being useful. We handed things off without getting that sorted out, which is why we felt it would be difficult to scale.

On top of that, this was before Takahiro had joined, so we didn't have test design expertise to begin with. VNEXT wasn't in a position to lead in that area either, and we felt we'd hit the limits of what we could do on our own — that's what led us to reach out to Verisurv. Once Verisurv came on board, they reviewed things from the ground up, starting with the fundamental question of how to approach building test cases, and that knowledge accumulated within the VNEXT team.

Takahiro: We ourselves didn't have deep expertise in MagicPod at that point either, so the guidance from Verisurv was genuinely educational. They taught us things on a near-daily basis — such as how to restructure test procedures to parallelize multiple test cases and shorten the execution period, and how to ensure test case updates are made reliably without anything slipping through.

Shortly before now, we expanded the automated testing team within VNEXT from 2 members to 4. The knowledge that the original 2 had gained from Verisurv was passed on smoothly to the new members, and they got up to speed without any significant issues. The fact that knowledge is now properly handed down is something I think is directly attributable to the hard work we put in during the initial setup period.

Nozomi: Were there any challenges on the product side in terms of element identification?

Mitsuhiro: PORTERS is built with React, so IDs are auto-
generated and can change, and users can freely configure screen fields, meaning what appears where can change day to day. But MagicPod handles this well — it's working without issues. We haven't had to ask the development team to make any special accommodations so far.

Takahiro: Occasionally, we get a message saying, "There's an element we can't capture, so we'll handle it with coordinate-based targeting," but it happens very rarely.

---

---

Closing

Takahiro: I believe that regardless of the system, the testing man-hours required for regression testing are never trivial. Automated testing is a highly effective solution for addressing that. Getting from zero to one is challenging, as we've discussed. But once you reach that point, it becomes an incredibly powerful asset. MagicPod in particular is easy to use and has excellent maintainability, so I can recommend it with confidence. It's been a huge help for us, and we hope to continue using it for a long time to come.

Mitsuhiro: We had our struggles with automated testing in the past, but now Takahiro manages it and VNEXT's dedicated team runs it every day. Being able to build this structure of "always keeping it running" has been truly significant. Automated testing will always produce errors, so the key is checking daily and keeping it in a consistently working state. That's what makes it actually succeed. I shudder to think about where we'd be without it now.

Being able to run tests as part of everyday processes rather than only before releases has allowed us to treat testing not as a special activity, but as an integral part of development. In the SaaS world, working software is a baseline expectation — if there are defects, customers simply can't use the product. Being able to ensure quality through testing and release with confidence is, I think, enormously valuable.

Reference:
Verisurv Case Study Interview
PORTERS Co., Ltd. case study interview now published

PORTERS Co., Ltd.

• Corporate site: https://www.porters.jp/
• PORTERS Tech Blog: https://zenn.dev/p/porters_tech

---

---

---

---

(Photo caption, left to right)
• Akari Idei — Consumer Services Company, First Product Design Division, Service Infrastructure Team
• Chikara Mitsui — Consumer Services Company, Senior Principal Architect
• Fumitaka Ueda — Consumer Services Company, First Product Design Division, Video Services, Second Technical Development Team
• Nozomi Ito — MagicPod CEO

---

Chikara : I serve as Senior Principal Architect across both the First and Second Product Design Divisions. I was originally the head of the Product Design Division, but now I take on a technical consulting and direction-setting role that spans both divisions.

DOCOMO's business runs on three pillars — telecommunications, Smart Life and enterprise, and our First and Second Product Design Divisions are responsible for developing and operating the Smart Life products.

Of the 100+ products we handle, about 60% are developed using agile methodologies, and E2E regression testing tends to become a bottleneck.

So when Akari's Base App team introduced MagicPod around May of last year as part of a test automation initiative and got solid results, and then the large-scale Lemino QA team also saw strong outcomes, we shared those findings at an internal tech-sharing session. Since then, we've been getting inquiries from teams beyond those two saying they want to adopt MagicPod as well.

Akari : I work in service infrastructure at the First Product Design Division. Our team doesn't deliver services directly to end users — instead, we develop internal platforms that provide shared functionality across various services and apps.

Within that, my team develops the "Base App," which allows service websites to be turned into apps quickly and at low cost. It comes with DOCOMO-specific features like authentication and push notifications built in, and can be customized relatively easily — currently deployed across seven services.

I serve as the product owner. I originally joined NTT DOCOMO Solutions (formerly NTT Comware) as an engineer working on DOCOMO-facing development. Around 2022 I transferred to DOCOMO and took on my current role. MagicPod was my first experience with automated testing — until then, we'd been doing E2E and regression testing entirely by hand.

Fumitaka : I joined as a new graduate in 2024, and from my very first year I've been serving as product owner on the video playback library development team. I got involved with Lemino starting in January 2025 — it was a moment when the team was looking to push test automation forward, and I was invited to join and raised my hand to participate.

As a product owner, I'd been involved in test design, but the actual automation had been left to the development team, so I didn't have any expertise myself. I was genuinely starting from zero, but now I lead the test automation team.

Nozomi: On the development structure — with this many products, I imagine you've also pushed quite far on in-house development?

Chikara: We only have a few hundred employees, yet we have over 100 products, so it's simply not possible to staff all development with our own people — even if everyone were a product owner. The basic model is to work with partners under contract-based arrangements, developing together as one integrated team. The Lemino team, for example, does have DOCOMO employees who write code, so I often use the term "semi-in-house" to describe it.

---

---

Challenges Before Adopting MagicPod

Akari: Our team handles everything from development through operations and maintenance with a single team, which made it difficult to allocate resources to testing. There were two main challenges.

The first was rework caused by running tests at the end of the process. Ideally, we'd run regression and device variation tests for each PBI (Product Backlog Item), but in practice the workflow had evolved into batching multiple sprints together and testing in the later stages. As a result, bugs from earlier development — including device- and OS-specific issues — would all surface at once, sometimes causing significant rework.

The second was development slowdowns from manual testing. The Base App is used across many DOCOMO services, so development speed is critical, but the effort required for manual testing kept becoming a bottleneck.

Nozomi: Wasn't there some benefit to batching the tests together?

Akari: Absolutely — doing everything at the end does reduce effort in a sense. But if a bug surfaces during testing after two months of development, and it traces back to something implemented at the very start, you end up with major rework that inflates the total effort anyway. There was also a growing sense of unease within the team about saving all the testing for the end, and building confidence in the process was a real challenge.

Fumitaka: In Lemino's case, there are multiple development teams alongside a separate QA team, and improving development speed while reducing costs were the big themes. As we pushed toward agile, the testing phase was the particular bottleneck — all E2E testing before a release was done manually, making the effort and costs enormous. Concretely, when bugs were found late in testing, there wasn't enough time to fix them before the release date, and the release itself would get pushed back.

Chikara: The Lemino flow was to batch multiple sprints of development, run E2E regression, then release — and MagicPod has been compressing that cycle. Akari's Base App team has gone even further, building regression testing into the development process at the PBI level and running it continuously — a much closer approximation of the ideal. When you catch a bug, you can fix it right away, and business agility goes up.

For a large service like d-Barai, you could bring in specialists and automate testing with Appium, but asking every team to shoulder that kind of maintenance burden isn't realistic. That's why we're using the results from Akari and Fumitaka's teams as success cases to drive MagicPod adoption more broadly.

---

---

Why They Chose MagicPod

Chikara: Before tools like MagicPod came along, teams working on mobile app automation had basically one option: Appium. Then no-code tools started appearing, and there was a sense that things could be done much more easily.

For DOCOMO, the first question was always whether a tool could meet our security regulations. We tested a few tools that cleared requirements around local execution and password handling, but MagicPod was far and away the strongest in terms of usability across both browser and mobile.

Akari: Our team also evaluated several tools including Appium, but they all fell short on cost or resource requirements, so MagicPod was the only one we actually trialed. We started the trial around May of last year and moved into production use by July.

Beyond cost, key selection factors were that the UI was intuitive from a product owner's perspective, and that we were already using GitHub as our repository, so being able to integrate with GitHub Actions for CI/CD was significant. Being able to set up a flow where automated tests run and results are ready the moment development finishes was really valuable.

Nozomi: How much did the no-code aspect factor into the decision?

Akari: Since the development team members are the ones actually using it, the no-code aspect itself wasn't a huge priority. That said, being able to check test status through the GUI as a product owner is genuinely helpful — in that sense, I do appreciate the no-code benefits.

Also, the Base App makes heavy use of WebView, so the fact that MagicPod handles WebView without issues has been a real help.

Fumitaka: For Lemino, we compared three tools including MagicPod. When we did a rough cost estimate for Lemino's scale, MagicPod came in at roughly one-quarter of the competition — a significant gap.

On the feature side, support for local execution was important since we want to test under conditions as close to the actual user environment as possible. Our targets include the mobile app, browser and TV-based platforms. Excluding TV, MagicPod was the only tool that covered every combination — local or cloud, mobile app or web browser.

The Lemino app is built in Flutter, and the trial revealed two challenges. One was that Flutter's architecture caused multiple widgets to be recognized as a single UI element, making it hard to reliably tap specific buttons. The other was that locators differed between Android and iOS, meaning we'd need separate test cases for each.

However, the MagicPod help documentation laid out a solution: by using Flutter's Semantics widget to assign unique IDs to each UI element, you can use a common locator for both Android and iOS and cover everything with a single test case.

Nozomi: Was the development team cooperative? I often hear that QA teams run into resistance when making requests of developers.

Fumitaka: Since assigning IDs is development work that isn't directly tied to functionality, I think the initial reaction was lukewarm. So we shared the problem and the expected benefits with the development team, making the case that test case maintenance effort could be cut roughly in half. That explanation got them on board, and from there the process moved to the development team proposing ID assignment plans that we would then review.

Now, whenever a new screen or feature is added, the development team proactively discusses the need for ID assignment during agile refinement sessions and handles it autonomously.

Nozomi: That's impressive. Was getting internal approval for MagicPod straightforward?

Akari: We presented the business case based on our trial results. Automation was already an area the entire Product Design Division had committed to advancing, so there was no pushback and the adoption went through smoothly.

Chikara: I had also been consistently telling people at internal tech-sharing sessions that without automated testing, no matter how much you optimize everything else, E2E testing will remain the bottleneck.

Through all of that advocacy, one thing that stood out — beyond just the low adoption cost — was the positive impact on quality. We started with delivery speed and cost efficiency as the goals, but quality improved as a result too.

With a large app, a specialist might spend one to two days determining the scope of regression testing and narrowing down which areas to cover, and occasionally that judgment call is wrong. With automated testing you can just run everything without narrowing scope, and since the number of test executions goes up, you catch bugs that would never have surfaced manually. That actually happened with Lemino.

At the "In-House Development Summit" I attended last year, more than half of the companies presenting had the MagicPod icon in their slides. That gave me real confidence we were heading in the right direction. People who have ownership over their tooling choices tend to make trustworthy ones.

---

---

How They're Using MagicPod

Akari: By automating regression and device variation tests that previously required a lot of manual effort, we've reduced testing effort by around 50%. We've achieved real shift-left — being able to redirect the time freed up by automation back into development has been a big deal.

Fumitaka: As of January 2026, nearly all automatable items in our app and web browser regression testing have been automated. With MagicPod we can run tests at roughly ten times the speed of manual testing, which has meaningfully accelerated the cycle of catching bugs early and fixing them before release — and we've also managed to catch bugs that would have been missed manually.

In our most recent mobile app release, we ran approximately 1,800 test cases and brought test execution effort down from 23 person-days to 3 person-days.

Chikara: Because Akari's setup is contained within a single team, the efficiency gains from testing flow directly back into development. In Lemino's case, where QA is a separate team with partner testers, the reduction in effort translates directly to reduced partner costs — a different flavor of the same benefit. Either way, the math shows that adoption costs pay for themselves within three release cycles, which makes it a particularly strong fit for scrum teams with high release frequency.

Fumitaka: Operationally, we run regression tests before each release and push results to Slack so the entire QA team can review them. The main focus is on failed cases — we analyze the cause, and if it's an app-side bug, we escalate to the development team to decide whether to fix it. Failed cases that turn out to be genuine new bugs are actually quite rare across the full regression suite, so escalations to the dev team don't happen all that often.

For Lemino, at the moment we're running from the command line rather than through GitHub Actions integration. The browser interface only lets you run one case at a time, but the command line allows parallel execution, so that's what we use. CI integration is something we want to pursue going forward.

Nozomi: On the topic of testing on real devices — have you looked into integrating with NTT Group's Remote TestKit?

Chikara: It's come up before. We see a MagicPod integration as something we'll definitely need eventually, and there are teams that have bought large numbers of physical devices for testing, so we'd love to expand to those teams as well.

Nozomi: Connecting MagicPod to Remote TestKit is as simple as switching the execution target — no extra setup. It eliminates the need to physically connect devices each time, and it's also a solid solution for increasing the volume of test runs. If you're interested, please feel free to reach out to our CS team.

---

---

---

Using the Test Automation Agent "MagicPod Autopilot"

Fumitaka: More recently we've been using MagicPod Autopilot when creating test cases. Concretely, we take the test items from Lemino's test suite, create test case information and a prompt to pass to Autopilot, feed that in and let it generate the test cases. The goal is an operational model where team members only need to review and adjust what the AI has produced.

We started out with around 20 prompt templates organized by test item type, but creating those templates was taking a lot of time in itself, so the next step we're working toward is automating the prompt creation as well.

Nozomi: What's the most important benefit you're expecting from Autopilot?

Fumitaka: The biggest one is reducing test case creation time. Beyond that, we're also thinking about whether integrating with the MCP server would let us run Autopilot in parallel — if so, the speed gains would increase further. On the quality side, test cases created by people inevitably have inconsistency in quality, and we're hopeful that Autopilot can bring them to a more consistent baseline.

Chikara: Looking at Autopilot's impact in numbers: initially it produced a 21% reduction in test case creation time. As we developed a better understanding of Autopilot's tendencies and refined our prompt templates, that rose to 31%. Then, when we started using Cline and the MagicPod MCP server to automatically generate prompts informed by existing test content and test perspectives, we reached a 66% reduction.

Fumitaka: Honestly, the quality of the generated test cases does vary — some only need one or two steps of adjustment, while others need a full overhaul before they'll run. But reviewing and adjusting is still vastly faster than building from scratch, so the efficiency gains are very real.

Chikara: We're also experimenting with combining Cline and the MagicPod MCP server to investigate the cause of test failures and suggest fixes. The accuracy rate is still around 50%, but we believe this kind of automated analysis will become essential going forward.

Nozomi: On MagicPod's end, we're actively developing a root cause analysis feature — the plan is that it will eventually allow you to make changes to related tests based on the analysis output. Modifying existing tests via the MCP server is already possible.

Chikara: Speaking of AI-based root cause analysis — this actually came up at a system monitoring event I spoke at recently. The idea was that combining MagicPod with the Datadog or New Relic MCP could be interesting: by cross-referencing test failure data from MagicPod with performance data from Datadog or New Relic, you could do root cause analysis at the full system level, not just at the application layer.

Nozomi: MagicPod doesn't capture performance data during test execution, so integrating with something like New Relic to layer performance information onto test results sounds really compelling. I think it opens up a world where you can see everything end-to-end, from client to backend.

Fumitaka: One question for you, Nozomi — tools like Playwright MCP have been emerging recently, where even people who can't write code can just give instructions to an AI and have tests created. Where does MagicPod's advantage lie in that landscape?

Nozomi: What Playwright MCP generates is code, so you need someone who can read code to judge whether it's correct, and modifications go through prompts, which tends to make the whole thing a black box. On top of that, there's prerequisite knowledge required — Git operations, setting up agent environments — which creates a high barrier to rolling it out across an entire QA team. The ability for a wide range of people to use something intuitively is where no-code tools continue to have real value.

Chikara: We actually have a specialist team that uses Appium and Playwright for production environment monitoring, but they're a self-contained team with people who can write code, so a code-based approach works for them. Asking everyone across all product teams — including people who don't write code — to do the same thing is just not realistic. That's precisely why I advocate for MagicPod.

---

---

In Closing

Akari: Adopting automated testing can feel like a high barrier, and the larger the service, the higher the adoption cost. That said, automated testing is an area that's only going to get more attention going forward, so I'd encourage starting small and expanding from there. That's exactly how our team was able to produce results.

Fumitaka: The strongest point of MagicPod, in my view, is the support. When evaluating tools, cost and features tend to get the most attention, but when you think about using something long-term, the quality of support matters enormously. There's an active Slack community where MagicPod and its users feel like they're on the same team. Seeing what's happening on the development side is also motivating for us. If you're trialing it, I'd really encourage you to reach out to support and get a feel for the MagicPod community.

Chikara: The genuine feeling is that it improves all of QCD — quality, cost and delivery. Usually improving one means sacrificing another, but automated testing is a trade-on, not a trade-off.

When I talk to people who've done automated testing, they all say the same things: "once you've done it, you can't go back" and "I don't understand why other teams aren't doing this." In the end, there's just an experience and information gap between those who have and those who haven't. The maintenance overhead is real, but what you get in return far outweighs it. My hope is that by leveraging MagicPod, teams can take the pain out of regression testing and keep growing into teams that deliver more and more value.


References
MagicPod User Meetup 2026 —
Chikara's presentation materials [NTT DOCOMO] Improving Development Efficiency with MagicPod at DOCOMO — and What Came Along With It
MagicPod Blog Award —
Special Prize Winner, Fumitaka's Blog How We Unified Android/iOS Automated Testing with MagicPod × Flutter and Cut Automation Costs by ~50%

NTT DOCOMO

• Corporate site: https://www.DOCOMO.ne.jp/english/corporate/
• DOCOMO Developer Blog: https://nttDOCOMO-developers.jp/

---

---

---

---

---

From left:
- Kiyosaki (QA Promotion Group, IT Infrastructure Department)
- Kurashige (Manager, Product Development Group 2, NX Development Promotion Department)
- Nozomi Ito (CEO, MagicPod)

---

Kiyosaki: I belong to the QA Promotion Group within the IT Infrastructure Department, where I am in charge of quality assurance for company-wide system development. Currently, the QA Promotion Group is a team of eight people, including external contractors.

Since joining RecoChoku in 2006, I have primarily worked as a Project Manager. During that time, I began to think, "How can we improve quality after release?", but because my work up to that point was unrelated to QA, I spent my days struggling how to tackle this matter. It was then that I was transferred to the existing QA organization, and I became involved in quality improvement efforts as a member of that team.

Initially, we conducted manual testing using human testers, but as no-code and low-code E2E tools began to appear on the market, I became interested, thinking, "Perhaps we can implement this ourselves." Currently, incorporating E2E testing into the QA team and deploying it horizontally across the entire company has become my main responsibility.

Kurashige: I joined the company in 2022 and have been responsible for the development of "d hits® powered by RecoChoku" and other services as a mobile app engineer. Currently, I serve as a manager in the department known as the NX Development Promotion Department. I was also thinking about "how we should operate testing to improve mobile app quality," so together with Kiyosaki, we are promoting initiatives for quality improvement through collaboration between engineers and QA.

---

Challenges Before Introducing MagicPod

Ito: What kind of challenges did you initially face regarding quality?

Kiyosaki: The challenge was that within the release cycle, we had a rule that "manual regression testing must be performed after fixes," which ensured quality assurance; however, every time the number of devices or OS versions to verify increased, the test patterns grew due to the combinations, leading to a shortage of manpower, so we had to secure external personnel to conduct manual testing each time.

In that situation, the final QA phase of the release took a long time, so calculating backward meant the development period became shorter, and if we tried to cut testing, the adjustments were difficult. The biggest issue was that the burden of testing was too great to keep up with the release speed, and it was something I worried about daily. We considered introducing in-house test automation, but due to the many difficulties involved, such as environment setup, finding personnel to maintain and operate it, and maintenance resources, we gave up on it.

Kurashige: As a result, at that time, significant man-hours were also being allocated on the development side, with development engineers performing unit tests, creating and executing visual regression tests, and manually conducting smoke tests for every build.

Kiyosaki: For QA as well, adjusting resources was incredibly difficult when multiple test requests came in simultaneously. Therefore, around 2020, we introduced a no-code/low-code automated testing tool for browser testing.

---

---

Reasons and History of Selecting MagicPod

Kiyosaki: In our company, there is a higher demand for mobile app testing than for browser testing, so I felt that we could not achieve true efficiency unless we automated mobile testing. It was at that time I happened to learn about MagicPod, and I felt it had the highest affinity with what we were seeking. The biggest deciding factor was the point that we "can test on real devices." Actually, that was a mandatory requirement for us.

The choice was between "building a real-device environment in-house" or "using a SaaS solution," and considering the effort required for environment setup and maintenance, building it in-house was not realistic for a small QA team. Therefore, relying on a SaaS platform was a prerequisite, but just when I thought there were no tools capable of realizing this, the tool I found was MagicPod. We confirmed there were no issues during the trial and decided to introduce it immediately.

Ito: You mentioned that testing on real devices is important; specifically, what kind of testing are you performing?

Kiyosaki: It is testing on devices that cloud simulators cannot handle. Specifically, based on our quality standards, we need to ensure that the app —even when no changes have been made—works properly on new devices and the latest OS/firmware, so testing this manually every time is very difficult. In that regard, MagicPod was extremely helpful because it allowed us to perform checks equivalent to manual testing using locally connected devices, and we could also automate the process.

Kurashige: The introduction of MagicPod happened right around the time I joined the company and started working on my first product development. Kiyosaki told me, "There is a tool like this," and when I tried touching it, I thought it was revolutionary. I felt, "This is something all mobile engineers should be able to use," and began evangelizing it within the company.

Development engineers also possess a sense of responsibility regarding quality, but balancing what to prioritize while advancing development can be difficult. I feel that having MagicPod has deepened the collaboration with the QA team, leading to successfully striking that balance.

Now, a natural flow has been established where we issue accounts to newly joined members, saying, "We automate with this, so please learn to use it."

Kiyosaki: Previously, when we introduced the browser test automation tool, QA took the lead, so we had regretted that it ended up being viewed as "something QA does." Since mobile app testing requires development engineers to build a dedicated app, the cooperation of development engineers is indispensable. Therefore, I thought it would be faster to have them see it working first, so I had Kurashige watch a demo.

---

---

MagicPod Utilization Status

Ito: You signed up for the mobile app plan in 2022, and in 2024, you migrated your browser plan to MagicPod as well.

Kiyosaki: When tools are separated, the management effort, such as learning costs, becomes significant. When we tried using MagicPod's browser testing, the usability was almost the same as the mobile version, so we decided to unify them. Since the scenarios used in the previous tool could be handled by MagicPod, the migration was extremely smooth.

Kiyosaki: Currently, we basically run daily executions across all services. Since there are more than 10 products, we stagger the execution times by 10-minute intervals while considering device resources. We distinguish between the main branch and feature branches; we run the main branch late at night with an image close to external service monitoring, while feature branches are executed during the day to link with development.

When there are dependencies in the execution order, such as tests spanning both browser and app, we utilize MagicPod's label management and schedule functions to ensure they run in the intended order, such as "execute browser at 15:00, execute app at 15:30."

The results are notified to Slack, distributed into two types: a channel summarizing all products and distributed channels viewed by each product representative. When an NG (failure) occurs, QA first isolates the issue, checking "is it a server-side problem?" or "is it caused by an app fix?", before escalating it. We are always conscious of test stability so as not to become "The Boy Who Cried Wolf."

Ito: How is the collaboration between Development and QA conducted?

Kiyosaki: We hold a meeting once a week to track KPIs and discuss testing for new features. For example, regarding the issue that "tests are unstable," we sometimes ask engineers, "Please fix this so the locator can be specified uniquely."

Communication has increased dramatically with MagicPod as the hub, and the development side has started to show concern, asking, "Won't this fix break the scenario?", allowing us to grasp the future development status earlier, which enables QA to prepare in advance.

As a result of these improvements, we initially aimed for a pass rate of 80% at the time of introduction, but now we are able to maintain about 90%. The remaining 10% is mostly flaky tests due to communication environments or server response states, rather than product defects.

Kurashige: The development side is also building a workflow aiming for a state where scenarios are corrected the moment development is completed. We are still proceeding with improvements to this "development and testing cycle" together with QA, utilizing AI and other technologies. I feel it is a very significant achievement that MagicPod has become a catalyst for development engineers to seriously think about "what quality assurance is" and "how to design tests."

---

---

Eliminating "Manual Double Management" with Branching Features and Accelerating CI Integration

Ito: Within the development flow, are the tests for feature branches integrated with CI/CD tools?

Kiyosaki: Yes, we are gradually advancing the integration. Our test execution triggers generally follow two systems, as shown in this workflow diagram.

---

---

First, the left side of the diagram is the "developer-initiated flow" for each project. At the time when code is merged into the develop branch, GitHub Actions activate and execute the test for the MagicPod feature branch. If the test fails, feedback is sent to the developer, and the correction cycle rotates.

And the right side of the diagram is the "QA-initiated flow." Here, from a separate repository managed by QA, triggers for periodic execution, such as once a week, are set via GitHub Actions. Through this, the test for the feature branch is executed via the MagicPod API, creating a system where the QA side can also periodically check quality.

Kurashige: This is the current workflow premised on the "concurrent development and testing" we discussed earlier. Theoretically, we aim for a state where testing is also completed at the time when development is finished, and code freeze begins.

The branch configuration recommended by MagicPod is highly compatible with our development flow. We also position MagicPod's main branch as the "standard branch for the development environment," which runs automatically every day via CI tools.

If an engineer creates a feature branch for a new function, they can proceed with test preparation for the new UI without affecting the production environment or the test cases of the main branch. Since we can specify the branch even via command line execution from GitHub Actions, it was smooth to incorporate into our process.

Ito: How did you manage things before the branching feature was released?

Kiyosaki: It was exactly "manual double management." We duplicated the entire scenario and applied corrections to the duplicated one as a feature branch. However, this manual duplication management always carried the risk of affecting other people's work. Therefore, thanks to the branching feature, each person can add and verify changes in an "independent workspace," and I feel the greatest benefit is that the team can collaborate safely and rapidly.

In terms of man-hours, it became much easier, and CI integration became easier as well. It was truly a long-awaited feature. Previously, since we only had the single main branch, failures in a feature branch under development would affect the health score. We are also helped by the fact that the stability of the main branch has become easier to visualize now that the branches are separated.

---

---

Conclusion

Kiyosaki: When implementing MagicPod, we tend to make "cost reduction" or "solving labor shortages" the objective. I myself initially focused too much on the point that I "wanted to reduce the cost of manual testing." However, I was made to realize through personal experience that "simply introducing a tool will not achieve that objective."

The most important thing in automation is not just calculating ROI (Return on Investment) first, but first building a "structure where automated testing can be executed stably and repeatedly." I believe this is the most essential first step.

Once that structure is established, tests will run stably, and cost reduction will definitely follow later as a "byproduct." Therefore, I recommend first firmly setting the axis of your objective and starting by involving development engineers to incorporate testing into the development cycle.

Tools that do not fit well will eventually stop being used. It is truly a waste if a convenient tool is introduced but ends up unused.

Kurashige: I have almost the same opinion. Quality assurance is indispensable to manufacturing, and the goal is "to maintain high quality." I think MagicPod is the perfect tool for that. Since I often develop on the front lines, I engage with the mindset of "changing the development process itself for the sake of quality assurance." If you "just put it in" or "just automated it," it won't last after all. I believe it is highly effective for teams that have the resolve to change their entire operation. I definitely recommend it.

---

RecoChoku Co., Ltd.

• Corporate Site: https://recochoku.jp/corporate/
• RecoChoku Engineer Blog: https://techblog.recochoku.jp/

RecoChoku develops music-related services by leveraging the latest IT technologies. This development blog shares knowledge gained from their daily activities.

• Recruitment Information: https://recruit.recochoku.jp/

RecoChoku is recruiting colleagues to work with. If you are interested, please visit their recruitment page!

---

---

---

---

---

From left to right:
- Ms. Miho Hayakawa, Team Leader, IT Product Development Team, Group Digital Solution Center
- Mr. Taizo Kawada, IT Product Development Team, Group Digital Solution Center
- Nozomu Ito, MagicPod CEO

---

Ms. Miho Hayakawa (Hayakawa): I serve as the Manager and Product Owner for the IT Product Development Team. Our team is part of the Group Digital Solution Center (GDSC), which has about 100 members in total. In the past, each brand had its own information systems department, but they were all integrated into the GDSC in April 2022 to maximize synergy across the group.

Within the GDSC, our IT Product Development Team consists of about 10 people, including both employees and contract members. We used to outsource all of our system development, but to keep up with the rapid pace of the market, we launched an in-house development team in January 2023. We started by bringing the development of Orbis's systems in-house. Because of that history, a lot of our work is still related to Orbis, but we aren't a brand-specific team; we're set up to work across the entire group.

Mr. Taizo Kawada (Kawada): My name is Taizo Kawada, and I'm the Scrum Master for the same IT Product Development Team. I led the effort to research, select, and implement MagicPod. At my previous job, I had seen firsthand the benefits of having automated testing—and the headaches that come from not having it. So, when I joined this company and saw that all testing was being done manually, I knew we had to do something about it, and I began looking into automation.

Ito (MagicPod CEO): Could you tell me a bit more about the specific systems you're working with now?

Kawada: The main system we're handling right now is for internal use, specifically the one that manages products for Orbis. This system's features were originally part of a massive, core legacy system that had been built up at Orbis over many years. The problem was that it was difficult to quickly improve its functionality to keep up with market changes. So, our team carved out the parts that required frequent changes and rebuilt them from the ground up. Our tech stack is Python for the backend and React for the front end.

---

The Challenges Before MagicPod

Kawada: I joined right after the in-house development team was formed, so the organization was brand new and the structure wasn't really solidified yet.
At the time, we were using Cypress for testing, but tests weren't being run automatically or on a schedule, and we couldn't keep up with maintenance. It was effectively non-functional. As a result, we had to rely on manual testing to verify that anything worked.

One of the biggest struggles was that the product management system had two distinct use cases: one for Orbis employees on PCs, and another for our in-store Beauty Advisors (BAs) on iPads. This meant we had to run tests in a normal browser and run separate, iPad-equivalent tests using Chrome's mobile emulation.

On top of that, we were still in the early stages of building our culture as a development organization. We all recognized the need for automated testing, but we just weren't making any headway. It was obvious that as we added more features, our testing time would only increase, so we started looking for an automation tool.

Ito: Do you have a dedicated QA team for testing?

Hayakawa: We don't have a QA team, so the development team handles everything from implementation to testing.

Kawada: Based on my past experience, I knew from the heart that automated testing is essential for delivering value in an agile way. I explained to Hayakawa, "If we don't tackle this now, it will absolutely become a bottleneck for our development speed down the road," and so we moved forward with automation.

---

---

Why and How We Chose MagicPod

Kawada: As I researched various automation tools, I narrowed our list down to three final candidates, including MagicPod. As I mentioned, we had existing test code, but it was broken and we didn't have the resources to maintain it. So, we prioritized ease of use and focused our search on no-code tools.

Ito: When you were narrowing it down to those three, what were your main criteria?

Kawada: I had never used a no-code testing tool before, so at first, I wasn't sure what to look for. While I knew we'd have to try them out to make a final decision, switching tools after you've already committed is a huge pain. I figured that a tool with a solid track record in the industry would be more mature and functionally polished, so I prioritized finding a well-established solution.

Ito: That's a priority we hear from a lot of companies. So, out of those three, what ultimately made you choose MagicPod?

Kawada: I saw a lot of case studies for MagicPod on sites like Qiita and Zenn, and the fact that MagicPod hosts its own seminars and has a user community gave us a lot of confidence. Ultimately, the deciding factors were that there's no limit on the number of test runs and that the monthly cost was reasonable, with single-month contracts available, which let us start small.

Other tools kept their pricing private and required you to contact them, which was a hurdle. But MagicPod had its pricing clearly laid out on the website, so I could immediately see that it was something we could try. That's when I requested a trial.

Ito: And how did you find the trial?

Kawada: What really stood out was how I could use most of the features intuitively, without overthinking it. I got it up and running after just a few questions with the sales representative, which made me think, "If I can use this, so can the rest of the development team." I rarely felt stuck wondering how to do something I wanted to do, and my lasting impression was that it was just incredibly user-friendly.

Ito: Thank you so much!

---

How We're Using MagicPod Today

Kawada: Currently, we are running two types of tests on our product management system: standard browser tests and iPad-equivalent tests using Chrome's mobile emulation. Specifically, we run a set of essential happy-path tests every hour between 9 a.m. and 6 p.m. on weekdays. The results are sent to a Slack channel, and our process is that if an error persists, the development team investigates.

Looking ahead, we're preparing test cases with the goal of running a full, comprehensive test suite once a day. Eventually, we want to take it even further by integrating it with our CI/CD pipeline to verify our most critical happy paths before every deployment.

Ito: Have you seen any measurable effects or time savings since the introduction?

Kawada: Since the iPad use in our stores is limited to simple read-only functions, we can now cover it completely with MagicPod. Thanks to that, the time we spent on manual verification—which used to be two to four hours per two-week sprint—has been reduced to zero. Overall, we do spend time creating and maintaining tests in MagicPod, so the net time savings are modest for now, but we've definitely felt that our hourly regression tests help us catch impacts on existing features much faster.

Ito: About how many people are using it at the moment?

Kawada: We've made it accessible to everyone on the IT Product Development Team. Not everyone uses it every day, since some of our products don't use MagicPod, but our rule in Scrum is that whoever picks up a task is responsible for making any necessary fixes. In that sense, you could say everyone is an active user.

Ito: What aspects of it do you feel are the best fit for your team?

Kawada: The fact that there are no limits on test runs or the number of users is incredibly convenient. I also love that you can create tests intuitively with no code and that the interface is so simple. There's a rich set of commands, and the ability to create shared steps for reusability and maintainability has been a huge help.

Ito: We're glad to hear it! By the way, have you had a chance to try our AI agent, "MagicPod Autopilot"?

Kawada: Yes, we've been experimenting with it. We're always talking about how we need to use generative AI to become more efficient, and we hope you'll continue adding features that help with test generation and reduce maintenance time. Our company is already moving forward with tools like GitHub Copilot and Devin, so we're hopeful that the MagicPod MCP server (CLI) will be enhanced to create a seamless, one-stop workflow from development all the way through testing.

Ito: That is definitely something we want to work on.

---

---

From "Is This Worth It?" to a Culture Where Passing Tests is the Norm

Ito: Beyond the day-to-day operations, have you noticed any changes in your team's culture or development process?

Kawada: I think it's had a really positive impact on the team's mindset. When I first joined, everyone knew that having tests that weren't maintained, weren't run on a schedule, and wouldn't even run when you tried was a bad situation. At the same time, because it would take so much work to get them running correctly, a negative vibe started to creep in—people would say "we'll get to it someday" or even "is there any point in doing this at all?"

Once you get into that state, the priority of fixing automated tests just keeps dropping, and it leads to a vicious cycle: automated tests aren't maintained → the tests fail → no one sees any benefit → motivation to maintain them disappears.

But we could see that as the product grew, the time spent on manual testing would also grow, leading to a clear drop in development speed and output. I knew we had to break that cycle somewhere. That's when I thought that by using an approachable, no-code tool, the team could experience the benefits of test automation for themselves.

And in practice, once we started with just a few test cases and gradually expanded our coverage, getting automated tests running in MagicPod, people started to see the benefits. A new attitude emerged, where they'd say, "This is failing, we need to look into it." Today, I feel our mindset has completely shifted to: "Tests are supposed to pass, and if they're failing, we need to fix or improve something."

Ito: Do you think that kind of vicious cycle can be broken by simply implementing a tool like MagicPod? Or is it more important to start by changing the mindset first?

Kawada: That's a tough question. I think it's very difficult to change a mindset without tangible results. It might be different if it's a top-down mandate to "just do it," but my personal view is that for things like this, if the people actually doing the work don't feel the impact, it's hard to make progress and nothing really changes. I'm sure every team has its own way of doing things, but I feel that you need something you can experience firsthand to get started. And as an option for that, I think MagicPod is very effective.

In terms of a concrete approach, I'm sure that at other companies, too, there are parts of their products where they think, "This area has a lot of bugs," or "We're constantly making fixes over here." I think starting in those places is a great way to see results quickly.

---

---

Final Thoughts

Kawada: To maintain quality while responding quickly and flexibly to rapidly changing market needs, I believe streamlining the testing process is absolutely essential. While you can't automate everything, I think MagicPod is a powerful option for front-end and E2E testing. We ourselves started with just one or two test cases and gradually expanded from there. The key to success, I believe, is to not aim for perfection from day one, but to start small.

In our case, bringing in MagicPod changed our perception of testing from a "chore we had to do" to an "important process for protecting quality." For anyone looking to start small, MagicPod's pricing structure makes it very approachable. Plus, the extensive Japanese-language support and community are reassuring, as it's easy to find information when you get stuck.

I imagine the challenges we faced are a common story that many companies can relate to. Of course, for a tech-focused company, testing might be second nature, but I suspect there are many companies out there that can't do it effectively due to a lack of resources. As features grow and services mature, the amount of testing work snowballs. To "automate what can be automated and spend your time on what truly creates value," I believe automation is something you absolutely have to do.

---

POLA ORBIS HOLDINGS INC.

• Engineer Recruiting Site：https://engineer.po-holdings.co.jp/
• Qiita：https://qiita.com/organizations/POHD

---