Documente Academic
Documente Profesional
Documente Cultură
Petr
HOW TO WRITE BETTER TESTS? Kainulaine
If you are struggling to write
≡ Menu
automated tests that embrace
change, you should find out how my
testing course can help you to write
tests
Are youfor Spring
tired and Spring
of writing Boot have a lot of boilerplate code? If so, get started with
tests which
applications. Spock Framework >>
The previous parts of my Spring Batch tutorial provided an introduction to Spring Batch and
described how we can get Spring Batch dependencies by using either Maven or Gradle.
After we have downloaded the required dependencies, we can start writing Spring Batch jobs.
The first thing that we have to do is to provide the input data for our batch job. This blog post
helps us to read the input data from CSV and XML files.
If you are not familiar with Spring Batch or Gradle, you should read the following blog
posts before you continue reading this blog post:
Spring Batch Tutorial: Introduction specifies the term batch job, explains why you
should use Spring Batch, and identifies the basic building blocks of a Spring Batch
job.
Due to GDPR, we have published our new privacy policy.
https://www.petrikainulainen.net/programming/spring-framework/spring-batch-tutorial-reading-information-from-a-file/ 1/21
3/6/2019 Spring Batch Tutorial: Reading Information From a File
Spring Batch Tutorial: Getting the Required Dependencies With Maven describes
HOW TO
howWRITE BETTER
you can TESTS?
get Spring Batch dependencies with Maven.
When we start writing a batch job, our first step is to provide input data for our batch job. In this
case, we have to read student information from CSV and XML files and transform that
information into StudentDTO objects which are processed by our batch job.
The StudentDTO class contains the information of a single student, and its source code looks
as follows:
Let’s find out how we can read information from a CSV file.
1 NAME;EMAIL_ADDRESS;PACKAGE
2 Tony Tester;tony.tester@gmail.com;master
3 Nick Newbie;nick.newbie@gmail.com;starter
4 Ian Intermediate;ian.intermediate@gmail.com;intermediate
We can provide the input data for our batch job by configuring an ItemReader bean. We can
configure an ItemReader bean, which reads the student information from the students.csv file,
by following these steps:
1. Create a CsvFileToDatabaseJobConfig class and annotate the created class with the
@Configuration annotation. This class is the configuration class of our batch job, and it
contains the beans that describe the flow of our batch job.
2. Create a method that configures our ItemReader bean and ensure that the method returns
an ItemReader<StudentDTO> object.
1. Create a new FlatItemReader<StudentDTO> object. This reader can read lines from the
Resource that is configured by invoking the setResource() method.
2. Configure the created reader to read student information lines from the
Due to GDPR, we have published our new privacy policy.
data/students.csv file that is found from the classpath.
Close Read Privacy Policy
https://www.petrikainulainen.net/programming/spring-framework/spring-batch-tutorial-reading-information-from-a-file/ 3/21
3/6/2019 Spring Batch Tutorial: Reading Information From a File
3. Configure the created reader to ignore the header line of the CSV file.
HOW TO WRITE BETTER TESTS?
4. Configure the reader to transform the student information line into a StudentDTO object
If you are struggling to write
by using these steps:
automated tests that embrace
1. Split
change, the student
you should information
find out how my line into tokens by using semicolon (;) as a delimiter
character
testing course and configure
can help the names of each token. The names of these tokens must
you to write
tests for match
Springwith
andthe field names
Spring Boot of the StudentDTO class.
applications.
2. Ensure that that the mapper component creates a new StudentDTO object and sets
its field values by using the created tokens.
1 import org.springframework.batch.item.ItemReader;
2 import org.springframework.batch.item.file.FlatFileItemReader;
3 import org.springframework.batch.item.file.LineMapper;
4 import org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper;
5 import org.springframework.batch.item.file.mapping.DefaultLineMapper;
6 import org.springframework.batch.item.file.mapping.FieldSetMapper;
7 import org.springframework.batch.item.file.transform.DelimitedLineTokenizer;
8 import org.springframework.batch.item.file.transform.LineTokenizer;
9 import org.springframework.context.annotation.Bean;
10 import org.springframework.context.annotation.Configuration;
11 import org.springframework.core.io.ClassPathResource;
12
13 @Configuration
14 public class CsvFileToDatabaseJobConfig {
15
16 @Bean
17 ItemReader<StudentDTO> csvFileItemReader() {
18 FlatFileItemReader<StudentDTO> csvFileReader = new FlatFileItemReader<>();
19 csvFileReader.setResource(new ClassPathResource("data/students.csv"));
20 csvFileReader.setLinesToSkip(1);
21
22 LineMapper<StudentDTO> studentLineMapper = createStudentLineMapper();
23 csvFileReader.setLineMapper(studentLineMapper);
24
25 return csvFileReader;
26 }
27
28 private LineMapper<StudentDTO> createStudentLineMapper() {
29 DefaultLineMapper<StudentDTO> studentLineMapper = new DefaultLineMapper<>();
30
31 LineTokenizer studentLineTokenizer = createStudentLineTokenizer();
32 studentLineMapper.setLineTokenizer(studentLineTokenizer);
33
34 FieldSetMapper<StudentDTO> studentInformationMapper = createStudentInformationMa
35 studentLineMapper.setFieldSetMapper(studentInformationMapper);
36
37 return studentLineMapper;
38 }
39 Due to GDPR, we have published our new privacy policy.
40 private LineTokenizer createStudentLineTokenizer() {
41 DelimitedLineTokenizer studentLineTokenizer
Close Read Privacy Policy= new DelimitedLineTokenizer();
42 studentLineTokenizer.setDelimiter(";");
https://www.petrikainulainen.net/programming/spring-framework/spring-batch-tutorial-reading-information-from-a-file/ 4/21
3/6/2019 Spring Batch Tutorial: Reading Information From a File
43 studentLineTokenizer.setNames(new String[]{"name", "emailAddress", "purchasedPac
44 return studentLineTokenizer;
45 }
HOW TO WRITE BETTER TESTS?
46
47 private FieldSetMapper<StudentDTO> createStudentInformationMapper() {
If you are struggling
48 to write
BeanWrapperFieldSetMapper<StudentDTO> studentInformationMapper = new BeanWrapper
49 studentInformationMapper.setTargetType(StudentDTO.class);
automated tests
50 thatstudentInformationMapper;
return embrace
51 }
change,
52 } you should find out how my
testing course can help you to write
tests for Spring and Spring Boot
Additional Reading:
applications.
Let’s move on and find out how we can read information from an XML file.
1 <students>
2 <student>
3 <name>Tony Tester</name>
4 <emailAddress>tony.tester@gmail.com</emailAddress>
5 <purchasedPackage>master</purchasedPackage>
6 </student>
7 <student>
8 <name>Nick Newbie</name>
9 <emailAddress>nick.newbie@gmail.com</emailAddress>
10 <purchasedPackage>starter</purchasedPackage>
11 </student>
12 <student>
13 <name>Ian Intermediate</name>
14 <emailAddress>ian.intermediate@gmail.com</emailAddress>
15 <purchasedPackage>intermediate</purchasedPackage>
16 </student>
17 </students>
We can provide the input data for our batch job by configuring an ItemReader bean. We can
configure an ItemReader bean, which reads the student information from the students.xml file,
by following these steps:
https://www.petrikainulainen.net/programming/spring-framework/spring-batch-tutorial-reading-information-from-a-file/ 5/21
3/6/2019 Spring Batch Tutorial: Reading Information From a File
1. Create an XmlFileToDatabaseJobConfig class and annotate the created class with the
HOW TO WRITE BETTER
@Configuration TESTS?
annotation. This class is the configuration class of our batch job, and it
2. Configure the created reader to read student information from the data/students.xml file
that is found from the classpath.
3. Configure the name of the XML element (student) that contains the information of a
single student.
4. Ensure that the reader transforms the processed XML fragment into a StudentDTO
object by using JAXB2.
1 import org.springframework.batch.item.ItemReader;
2 import org.springframework.batch.item.xml.StaxEventItemReader;
3 import org.springframework.context.annotation.Bean;
4 import org.springframework.context.annotation.Configuration;
5 import org.springframework.core.io.ClassPathResource;
6 import org.springframework.oxm.jaxb.Jaxb2Marshaller;
7
8 @Configuration
9 public class XmlFileToDatabaseJobConfig {
10
11 @Bean
12 ItemReader<StudentDTO> xmlFileItemReader() {
13 StaxEventItemReader<StudentDTO> xmlFileReader = new StaxEventItemReader<>();
14 xmlFileReader.setResource(new ClassPathResource("data/students.xml"));
15 xmlFileReader.setFragmentRootElementName("student");
16
17 Jaxb2Marshaller studentMarshaller = new Jaxb2Marshaller();
18 studentMarshaller.setClassesToBeBound(StudentDTO.class);
19 xmlFileReader.setUnmarshaller(studentMarshaller);
20
21 return xmlFileReader;
22 }
23 }
Due to GDPR, we have published our new privacy policy.
https://www.petrikainulainen.net/programming/spring-framework/spring-batch-tutorial-reading-information-from-a-file/ 6/21
3/6/2019 Spring Batch Tutorial: Reading Information From a File
Before we can transform the student information read from the student.xml file into StudentDTO
HOW we
objects, TO have
WRITEto BETTER TESTS?
configure the name of the fragment root element in the TodoDTO class. We
canIf do
youthis
arebystruggling
followingto
these
writesteps:
automated tests that embrace
1.change,
Annotateyou
theshould
class with
find the @XmlRootElement
out how my annotation.
testing course can help you to write
2. Configure the name of the root element by setting the value of the @XmlRootElement
tests for Spring and Spring Boot
annotation’s name attribute to: student.
applications.
1 import javax.xml.bind.annotation.XmlRootElement;
2
3 @XmlRootElement(name="student")
4 public class StudentDTO {
5
6 private String emailAddress;
7 private String name;
8 private String purchasedPackage;
9
10 public StudentDTO() {}
11
12 public String getEmailAddress() {
13 return emailAddress;
14 }
15
16 public String getName() {
17 return name;
18 }
19
20 public String getPurchasedPackage() {
21 return purchasedPackage;
22 }
23
24 public void setEmailAddress(String emailAddress) {
25 this.emailAddress = emailAddress;
26 }
27
28 public void setName(String name) {
29 this.name = name;
30 }
31
32 public void setPurchasedPackage(String purchasedPackage) {
33 this.purchasedPackage = purchasedPackage;
34 }
35 }
Because the element names of the processed XML fragments are the same as the field
names of the StudentDTO class, we don’t have to add additional annotations into the
StudentDTO class. If this is not the case, or we need to use custom marshalling, we
have to annotate ourDue
DTO class with
to GDPR, JAXB
we have annotations.
published our new privacy policy.
https://www.petrikainulainen.net/programming/spring-framework/spring-batch-tutorial-reading-information-from-a-file/ 7/21
3/6/2019 Spring Batch Tutorial: Reading Information From a File
Additional Reading:
HOW TO WRITE BETTER TESTS?
If you The Javadoc package
are struggling to write summary of the javax.xml.bind.annotation package
automated tests that embrace
The Javadoc package summary of the javax.xml.bind.annotation.adapters
change, you should find out how my
package
testing course can help you to write
Lesson:
tests for Streaming
Spring API for
and Spring BootXML
applications.
Spring Batch Reference Documentation: 6.7 XML Item Readers and Writers
Summary
This blog post has taught us four things:
If we need to read lines from an input file, we can use the FlatItemReader<T> class.
The next part of this tutorial describes how we can read information from a database.
P.S. You can get the example applications of this blog post from Github: Spring example and
Spring Boot example.
https://www.petrikainulainen.net/programming/spring-framework/spring-batch-tutorial-reading-information-from-a-file/ 8/21
3/6/2019 Spring Batch Tutorial: Reading Information From a File
Your Email
SUBSCRIBE
RELATED POSTS
SPRING FRAMEWORK /
SPRING FRAMEWORK /
https://www.petrikainulainen.net/programming/spring-framework/spring-batch-tutorial-reading-information-from-a-file/ 9/21
3/6/2019 Spring Batch Tutorial: Reading Information From a File
Charles
HOW TOLink
WRITE BETTER TESTS?
June 24, 2016, 17:03
If you are struggling to write
automated tests that embrace
Being very (just a few days) new to Spring Batch, the first question I have is, once the classes
change, you should find out how my
described above are created, how do you run them to actually see the data defined in the XML
testing course can help you to write
file in the database? Thank you.
tests for Spring and Spring Boot
applications.
REPLY
Petri Link
I will describe the required steps in my upcoming blog posts, but basically you have to
follow these steps:
4. Ensure that the Spring container finds your component during classpath scan.
For example, if you want see the configuration of a Spring Batch job that reads information
from a CSV file and writes it to a database, you should take a look at this package. Also,
you might want to take a look at this resource page that contains links to other Spring Batch
tutorials.
REPLY
vk Link
https://www.petrikainulainen.net/programming/spring-framework/spring-batch-tutorial-reading-information-from-a-file/ 10/21
3/6/2019 Spring Batch Tutorial: Reading Information From a File
How to read/write multiple txt files (which are not same) using spring batch with single field
HOW TOclass?
setmapper WRITE BETTER TESTS?
If you are struggling to write
REPLY tests that embrace
automated
change, you should find out how my
Petri course
testing Link can help you to write
tests for Spring
September 5, 2016, and
22:20Spring Boot
applications.
Hi,
REPLY
noopur Link
REPLY
Petri Link
If you want to use Spring Batch, you need to create an ItemWriter that prints the
information to console. However, you probably don’t need Spring Batch for this. If you want
to use a simpler approach, take a look at this blog post.
REPLY
1. Do you have any samples with reading multiple FragmentRootElementName from XML file?
2. IHOW
haveTO WRITExsd
schema BETTER
file forTESTS?
the xml file I read in Spring Batch. According to your example I
have to annotate
If you the class
are struggling with the @XmlRootElement annotation. But I do not want to modify
to write
classes created
automated during
tests thatbuild process in the target folder.
embrace
change, you should find out how my
REPLY
testing course can help you to write
tests for Spring and Spring Boot
Petri Link
applications.
November 23, 2016, 17:04
Hi,
Do you have any samples with reading multiple FragmentRootElementName from XML
file?
No :(
I have schema xsd file for the xml file I read in Spring Batch. According to your example
I have to annotate the class with the @XmlRootElement annotation. But I do not want
to modify classes created during build process in the target folder.
https://www.petrikainulainen.net/programming/spring-framework/spring-batch-tutorial-reading-information-from-a-file/ 12/21
3/6/2019 Spring Batch Tutorial: Reading Information From a File
REPLY
Petri Link
Are you trying to insert something to the database by using Hibernate? If so, you need to
transform the StudentDTO object into an entity and persist it to the database. If the entity
has relationships with other entities, you need to obtain references to these objects before
you persist the created entity to the database.
REPLY
Rahul Link
Hi Petri,
I have downloaded the source code from Github. Please let me know how to run. Which is the
first class that needs to be executed?
REPLY
Due to GDPR, we have published our new privacy policy.
https://www.petrikainulainen.net/programming/spring-framework/spring-batch-tutorial-reading-information-from-a-file/ 13/21
3/6/2019 Spring Batch Tutorial: Reading Information From a File
Petri Link
HOW TO
August 22,WRITE BETTER TESTS?
2017, 18:17
REPLY
osama Link
How do i first fetch a csv file from any physical location then stream this csv file and read it in
spring batch.
if possible, could explain this without using annotation also.
REPLY
Petri Link
Hi,
Unfortunately Spring Batch doesn’t have a very good support for reading files from paths
that are not known when the application context is started. That’s why people often end up
writing custom tasklets which copies the input files to the path that is given to the used
ItemWriter.
https://www.petrikainulainen.net/programming/spring-framework/spring-batch-tutorial-reading-information-from-a-file/ 14/21
3/6/2019 Spring Batch Tutorial: Reading Information From a File
1. Write a tasklet that copies the input file to the known path.
HOW TO WRITE BETTER TESTS?
2. Create a step that invokes your tasklet and ensure that this step is run before the step
If you are struggling to write
that processes the input file.
automated tests that embrace
change, you should find out how my
If you don’t know how you can create custom tasklets, you should take a look at this blog
testing course can help you to write
post. Also, if you have any other questions, don’t hesitate to ask them.
tests for Spring and Spring Boot
applications.
REPLY
JH Link
Hi
what about nested elements ? I mean if a student is represented like that:
Tony Tester
Maths
221
Chemistry
222
tony.tester@gmail.com
master
REPLY
sorry xml is not supported, so here is my illustrated xml sample as I talked about in my
HOW TO WRITE
previous BETTER), TESTS?
post (without I hope It’s pretty clear…
If you are struggling to write
student tests that embrace
automated
name Tony
change, youTester
shouldname
find out how my
courses
testing course can help you to write
course
tests for Spring and Spring Boot
name Maths name
applications.
room 221 room
course
course
name Chemistry name
room 222 room
course
courses
emailAddress tony.tester@gmail.com emailAddress
purchasedPackage master purchasedPackage
student
REPLY
Petri Link
Hi,
By the way, if you have any additional questions, don’t hesitate to ask them.
https://www.petrikainulainen.net/programming/spring-framework/spring-batch-tutorial-reading-information-from-a-file/ 16/21
3/6/2019 Spring Batch Tutorial: Reading Information From a File
Petri Link
Hi,
Unfortunately I don’t have any such blog posts. However, I can help you to find one if you
provide additional information about your use case. For example, it would be useful to know
what kind of format is used by your input files.
REPLY
vRadhe Link
ok, i’ll send you my input file. can you send me any mail or fb id? it will be easy to send
file
REPLY
VRadhe Link
Petri , My use case is like that: for single record(person) i have multiple data and each
Due to GDPR, we have published our new privacy policy.
data have own identification code sush like
Close Read Privacy Policy
https://www.petrikainulainen.net/programming/spring-framework/spring-batch-tutorial-reading-information-from-a-file/ 17/21
3/6/2019 Spring Batch Tutorial: Reading Information From a File
REPLY
Leave a Comment
Name
Name (required)
Comment
Save my name, email, and website in this browser for the next time I comment.
SUBMIT
https://www.petrikainulainen.net/programming/spring-framework/spring-batch-tutorial-reading-information-from-a-file/ 18/21
3/6/2019 Spring Batch Tutorial: Reading Information From a File
Subscribe my email newsletter AND you will get an email when I publish a new blog
post
Your Email
Subscribe Now
JUnit 5 Tutorial
TestProject Tutorial
WireMock Tutorial
https://www.petrikainulainen.net/programming/spring-framework/spring-batch-tutorial-reading-information-from-a-file/ 19/21
3/6/2019 Spring Batch Tutorial: Reading Information From a File
Maven Tutorial
SEARCH
enter search term and press enter
https://www.petrikainulainen.net/programming/spring-framework/spring-batch-tutorial-reading-information-from-a-file/ 20/21
3/6/2019 Spring Batch Tutorial: Reading Information From a File
© 2010-Present Petri Kainulainen (all code samples are licensed under Apache License 2.0)
Sitemap | Cookie Policy | Privacy Policy
https://www.petrikainulainen.net/programming/spring-framework/spring-batch-tutorial-reading-information-from-a-file/ 21/21