How to install custom components into Talend Open Studio

How to install custom components into Talend Open Studio

Talend Open Studio is a handy ETL tool which amazing extending capabilities and a complete set of tools for building new custom components as I showed in several posts in the past. Talend offers an automatic way to install components through their official marketplace. However, that place is not famous for UX and for being attractive for developers and, as a matter of facts, components hosted there are often poor and outdated. Here I’m going to show a general way to install custom components which works for both Talend Exchange components and third-party hosted ones. For hard-core developers, I’ll also show how to compile a component starting from source code.

How to install a custom component package from binaries

  1. First of all, you need to create a folder into your local filesystem to store custom components. I generally create a folder named talend_components inside my Home directory, but the choice is up to you.
  2. A custom component you download from the Internet is usually packaged as a ZIP file. Unzip it in the folder you just created, inside a subfolder with the same name of the component itself (ie. tTwitterInput).
  3. Then, open Talend Open Studio and select a project from your marketplace, as usual. The standard perspective will show up, as usual.TOS - Job Design Perspective
  4. Open Talend Open Studio preferences from menu Windows -> Preferences. If you’re familiar with Eclipse, you’ll notice this is no more than the Eclipse Preferences Pane. But you’ll see two new voices: Talend and Talend Component Design.
  5. In Talend tab, locate the User component folder field and fill with the path of the folder you made at Step 1. This is the directory that Talend will look at startup to search for custom components to add to the Palette.TOS-Preferences
  6. At the same way, in Talend Component Design tab (Project Components), there’s just one field to fill with the same path. This is the directory that Talend Internal Component Designer will look to import and modify components. More on this later on this tutorial.
  7. Push Ok to apply new preferences and all the components you put on the folder at Step 2 will be automatically compiled and deployed to the Palette. If not, restart Talend. If still not, or if you simply want to understand better what’s going on under the hood, go on reading!

Please note that Preferences are Project-based. This means that if you switch or open another project, you’ve to fix the above properties again.

How to use push components from Component Designer

Something that very few ETL developers know is Talend is shipped with a rudimentary yet effective Component Designer. This is not the place to explain how to to bake a component (since I did elsewhere). But the Component Designer has the nice feature for “pushing” components to the Palette, so you don’t need to restart Talend any time you add/modify/remove a component.

I assume you followed all the Steps from the previous paragraph. And I especially refer to Step 6, since it’s the folder where the Component Designer will look inside to do its stuff. Then, let’s continue the tutorial.

  1. To switch to Component Designer, click on Window -> Perspective -> Component Designer. If you (like me) think you’ll often use the Component Designer, you should add a shortcut on the canvas. You can do this by clicking the “+” icon on the upper-right corner of the canvas (just over the Palette) and add the Component Designer. This will add a button to fastly call the Designer at any time, without the need of navigating the Menu. You’ll probably then need to resize that area a bit to make the label fit.TOS-perspective-shortcuts
  2. If you correctly set the paths on the previous steps, the Component Designer will automatically load all the components it finds. You can find them on the left pane. If you add/modify/remove a component from that folder while the Component Designer is still opened, press F5 or click “Update” from the contextual menu to refresh the view.TOS-Components Designer List
  3. Select a component, it doesn’t matter which one (you simply have COMPONENTS_PROJECT to not be selected) and select Push Components to Palette from the contextual menu. Doing that, Talend will recompile the components and made them available to the Palette. Please note that any components will be pushed/refreshed, not just the one selected. There’s no way to selectively push components, yet.
  4. Assuming that there weren’t compilation errors, if you still don’t see the component into the Palette, you probably have to delete the component cache. To do that, close Talend, then delete the file ComponentsCache.javacache from the subfolder configuration of the installation directory of Talend Open Studio. Then, restart Talend and see if this solves the problem.

How to build a component from source code

Sometimes you want to install a component directly from the source code. This is true, for example, for not yet officially released components, for snapshot versions and so on. As any good Java developer would say, there’s not a unique way to do that.

However, since I endorse the use of Maven as a way to build Talend components, I’ll explain the procedure for such those components that are mavenized by design. As a matter of facts, all the components you may find on this site are built using Maven, so you have plenty of examples.

Again, you need to perform all the previous Steps before going further.

  1. Download and install Eclipse for Java developers. Before you ask, yes, you can use your IDE of choice, instead. And yes, you can use bare maven command line interface only to do the task as well.
  2. Download and install Maven2Eclipse integration. To do so and in the majority of Eclipse distributions, you simply have to drag the Install button on the page into Eclipse.
  3. Download the source code for the component you want to install and unzip somewhere. Notice the pom.xml file which means, at the end of the day, that  the component is built using Maven. Here‘s a typical example.
  4. Now open Eclipse and from the File menu click Import…
  5. From the list of available project types, select Existing Maven Projects. You can type “Maven” into the text field to filter, if you have a lot of.Eclipse - Import Maven Project
  6. Fill with the root  folder where you unzipped the component source code before and confirm using default settings. Your component is a good-ol’ Eclipse project, now.
  7. Double click pom.xml to open it. Locate the Properties area.Eclipse - TOS Component parameters
  8. Double click to componentsFolder parameter and update it with the full path of the component folder you created at the beginning of this tutorial. By default, it points to a subfolder in your home directory called talend_components. Just to be clear, this is your destination folder.
  9. Select the pom.xml file and click Run as… -> Maven install from the contextual menu. The component will be automatically compiled and copied to the destination folder.Eclipse - Maven install action
  10. Restart Talend or follow the procedure for pushing the component to the Palette to start using the component.

Hope this helps! Have a nice day!

  • koceila

    Hello,
    Thank you Gabriele, your post is very usefull ;)

    • http://gabrielebaldassarre.com/ Gabriele

      Tnx allot! I’m happy to hear that!: :)