Cases of Talend Implementation in Enterprise Projects: Global Experience
Talend ESB is a software with a set of rules and principles that provides the possibility to exchange messages between different data systems and applications with the routing setup via a single point.
BANKING SECTOR
Modernization of the Banking System Using the ESB Integration
Landesbank is a German international universal bank that provides services in four major areas: private banking (VIP personal banking), corporate banking, commercial real estate loans, and selective capital market activities. The Bank serves about two million individuals and about 69 thousand corporate clients in more than 200 branch offices in Berlin.
Problem
Bank's IT System Obsolescence
To cope with the ever-rising account data volume, Landesbank decided to modernise its existing core banking system. However, the new system implementation required substantial advance preparation, and an unsuccessful changeover could actually bring the entire bank to a standstill.
Landesbank is a member of Germany's national savings banks group, and its business processes are closely connected to the organization's central IT provider Finanz Informatik (FI), that offers its own modern core banking system known as OSPlus. The complexity of updating the system was also due to the fact that in order to integrate its own banking applications for customer, address, revenue and account data, Landesbank needed to create an entirely new interface landscape parallel to OSPlus rollout.
Moreover, it was necessary to take into account a wide range of incoming data formats, and ensure the data is converted into the required formats to be further transferred to specialists in different departments that process documents in specific formats. The previous integration platform used by the bank was not up to the task of handling a massive data flow, and the transition to the new system could only be implemented in several stages, which was incredibly time-consuming.
Solution
Optimize information transfer processes
Talend ESB integration helped optimize information transfer processes. Now, when a business process is triggered in the OSPlus banking system, data is transferred to other applications via the Talend ESB. Most interfaces are event driven; only a few are scheduled.
To ensure that this process is also transparent within the company, dedicated TMF portal (Tools & Methods Framework) containing information and forms was created to monitor the company's error management workflow.
E-commerce
Simplification of Integration in the Online Store Infrastructure with ESB
Otto Group is one of Europe's most successful e-commerce companies and Germany's biggest online retailer with more than 2.8 million article positions and 6800 clothing brands for the entire family. OTTO processes about 7 million online orders annually and at peak periods it receives up to 10 new orders a second.
Problem
The Complexity of the Project's IT Landscape
The existing IT architecture of the online store consisted of a large Teradata database and many islands of data. The architectural complexity was also due to the fact that some data islands used SQL, some PL/SQL, some were Oracle-based, and some used PERL scripts. So, in order to understand what kind of advertising would be the most effective for clients, the company had to do a lot of data transformation, and the data system needed a single integration tool.
Moreover, the project had a lot of integration scripts, which were very difficult to maintain, because the original developers had already left the Company.
Solution
Data Lake, связанный с системами компании через Talend
Otto decided to build a completely new data lake and use Talend as a bridge between the new data lake, which stores data from the store website and from partner websites, and the company's 250TB Exasol database.
Talend software also made it possible to use company's own CI/CD process in the development. Talend also allowed OTTO to continuously integrate data sources and continuously develop and deliver new, small upgrades, updates, and applications as well as create test environments for new software launches. All this helped accelerate the process of development and troubleshooting, and allowed the overall reduction of risks associated with new developments and integrations.
So, if a customer wants his or her data to be removed from the OTTO system, the overall process won't take much time, since there is no need for searching several databases. Talend made it possible to communicate quickly with the data sources and simplify the process of deleting the data from Exasol and the Cloudera data lake.
PHARMACEUTICAL INDUSTRY
Process Automation, Processing and Storage of all Data in a Single System Using the Enterprise Service Bus
Bayer AG is a German multinational pharmaceutical and life sciences company with more than 150 years of history and a key competence in the field of healthcare in the world.
Problem
Outdated Information System with Manual Data Processing
Bayer carries out continuous research and testing of new drugs to determine their effectiveness and drug safety for further market launch. This requires the accumulation, analysis and storage of huge amounts of clinical findings from patients and volunteers. The data also comes from laboratories and electronic devices. All data is automatically anonymized at the collection point.
The previous information system was outdated and could not cope with the existing load and an ever-growing data flow; some processes were not automated and operations were processed manually, which often led to errors due to human factor.
Solution
Microservice IT-architecture
Instead of managing the old system, Bayer decided to completely change the environment and architecture of the infrastructure. Bayer hosted a "hackathon", when they performed a full-fledged Talend testing and came to the conclusion that it is a perfect solution for continuous system improvement, given its principles of flexibility and the ability to support a microservice architecture.
The Talend ESB bus has been used as a middleware that connects repositories and is not visible to users. The data is transmitted into Oracle's local data lake, and the company connects to the SAS system and uses the Oracle Life Sciences Data Hub to store data on clinical trials of drugs and medicines to perform advanced analytics.
The Talend ESB service bus made it possible to promptly add new functions and in accordance with the current requirements, to improve the quality of received data and to accelerate its processing. Another advantage of Talend use is its ability to track everything that happens with the data, including the data source and the destination.
Смотреть все
Обеспечили Polaris возможность легко выводить новые товары на маркетплейсы и менять информацию о товарах в несколько кликов
Подробнее
Смотреть все
YouTube
Собрали все грабли в интеграциях, чтобы не собирали вы
Смотреть все
Интеграции как индикатор качества вашей IT-архитектуры
Шины — ключевой элемент качественной архитектуры
Синхронность, асинхронность и при чем здесь шины
Talend ETL →
Универсальный инструмент для разработки микросервисов интеграций в философия low-code.
Kafka Streams →
Фреймворк для Kafka, обеспечивающий ETL слой без развитых low-code инструментов. Этот фреймворк позволяет брокеру выступать в роли сервисной шины (ESB).
Mule →
ETL/iPaaS решение, входит в семейство Salesforce. В философии no–code/low-code.
WSO2 →
ETL/iPaaS решение, входит в семейство Salesforce. В философии no–code/low-code.
DATAREON →
Универсальный инструмент для разработки микросервисов интеграций в философия low-code.
Apache Airflow →
Фреймворк для Kafka, обеспечивающий ETL слой без развитых low-code инструментов. Этот фреймворк позволяет брокеру выступать в роли сервисной шины (ESB).
Калькулятор интеграции систем (ESB)
0 рублей
0 рублей в месяц
0 часов
0 недель
Скопировать результат
1
Пример расчета
Для передачи данных между системами мы создаем "поток". Одни потоки нужны для отправки данных, другие - для получения. В отдельном потоке могут передаваться заказы, товары или другие сущности.
Для примера на схеме:
1. «Система управления товарами» отправляет товары. «Система управления складом» — факт отгрузки заказа. «Система управления заказами» — заказы. Итого системы будут отправлять 3 потока;
2. «Система управления складом» принимает товары и заказы. «Система управления заказами» — товары и факт отгрузки заказа. Итого системы будут принимать 4 потока.
2
Состав работ в калькуляторе
Включено в расчет
Дополнительно
Подготовка карты систем и потоков данных (SOA схема)
Подготовка инфраструктуры для работы коннекторов
Проработка логики объекта (схема бизнес-процесса коннектора)
Настройка контура мониторинга и логирования
Создание коннекторов для обмена данными по каждому потоку на 3 стендах (тест, препрод, прод)
Создание коннекторов (хранилище - получатель) для обмена данными по каждому высоконагруженному потоку (>100 сообщений в минуту) на 3 стендах (тест, препрод, прод)
Настройка до трех дашбордов на один коннектор внутри готового контура мониторинга
Свыше 15 атрибутов для каждого потока
Документация по копированию интеграции, переиспользованию, обслуживанию
Демонстрация реализованного функционала
Включено
Подготовка карты систем и потоков данных (SOA схема)
Проработка логики объекта (схема бизнес-процесса коннектора)
Создание коннекторов (источник - хранилище, хранилище - получатель) для обмена данными по каждому объекту на 3 стендах (тест, препрод, прод)
Настройка до трех дашбордов на один коннектор внутри готового контура мониторинга
Свыше 15 атрибутов для каждого объекта
Дополнительно
Подготовка инфраструктуры для работы коннекторов
Настройка контура мониторинга и логирования
Создание коннекторов (хранилище - получатель) для обмена данными по каждому высоконагруженному объекту (>100 сообщений в минуту) на 3 стендах (тест, препрод, прод)
Свыше 15 атрибутов для каждого объекта
Ваша заявка отправлена успешно
Отправить снова
You'll be contacted by your personal manager
Contacts
Make an appointment