The 7th International Workshop on Testing Database Systems (DBTEST) was held on

June 15th, 2018 in the morning.

within the arrangements of
2018 ACM SIGMOD/PODS to be held in Houston, TX, USA,
on June 10th - June 15th, 2018.




 The presentations of all speakers are available now.
 Videos from DBTest are online now on youtube

The workshop program is online.

Keynote:  DBTest 2018 will feature a keynote presentation from

Manasi Joshi, Google

Manasi Joshi is a Director of Software Engineering at Google. She is leading the ML Productivity effort with close collaborations across Tensorflow, Cloud, Applied Research and product engagement efforts in Google. Prior to this role, Manasi has a long tenure as an engineering productivity lead in Google’s Display Ads infrastructure team.

on   Machine Learning Testing and Productivity

Machine Learning is infused in all walks of life including a lot of Google products such as Google Home, Search, Gmail, and more, as well as in systems such as those used by self-driving cars and fraud detection systems. Google is an AI-first company and a tremendous amount of effort is being made to improve people’s experiences using Google products. However, developing and deploying high-quality, robust ML systems at Google's scale is difficult. This can be due to many factors including but not limited to distributed ownership, training serving skew, maintaining privacy and proper access controls of data, model freshness and compatibility. In the face of such challenges, we started an “ML productivity” effort to empower developers to move quickly and launch with confidence. This effort encompasses building infrastructure for reliability and reusability of software as well as extraction of critical ML metrics which can be monitored to make informed decisions through the ML life cycle. In this talk, we will discuss a few examples where these efforts may be applicable.
Our Sponsors


With the ever increasing amount of data stored and processed, there is an ongoing need of testing database management systems but also data-intensive systems in general. Specifically, emerging new technologies such as Non-Volatile Memory impose new challenges (e.g., avoiding persistent memory leaks and partial writes), and novel system designs including FPGAs, GPUs, and RDMA call for additional attention and sophistication.

Reviving the previous success of the six previous workshops, the goal of DBTest 2017 is to bring researchers and practitioners from academia and industry together to discuss key problems and ideas related to testing database systems and applications. The long-term objective is to reduce the cost and time required to test and tune data management and processing products so that users and vendors can spend more time and energy on actual innovations.

The workshop focusses on the following topics:

  • Testing of database systems, storage services, and database applications
  • Testing of database systems using novel hardware and software technology (non-volatile memory, hardware transactional memory)
  • Testing heterogeneous systems with hardware accelerators (GPUs, FPGAs, ASICs, …)
  • Testing distributed and big data systems
  • Testing machine learning systems
  • War stories and lessons learned
  • Performance and scalability testing
  • Testing the reliability and availability of database systems
  • Algorithms and techniques for automatic program verification
  • Maximizing code coverage during testing of database systems and applications
  • Generation of synthetic data for test databases
  • Testing the effectiveness of adaptive policies and components
  • Tools for analyzing database management systems (e.g., profilers, debuggers)
  • Workload characterization with respect to performance metrics and engine components
  • Metrics for test quality, robustness, efficiency, and effectiveness
  • Operational aspects such as continuous integration and delivery pipelines
  • Security and vulnerability testing

Paper Submission

Authors are invited to submit original, unpublished research papers that are not being considered for publication in any other forum.
Papers must follow the most recent ACM Proceedings Format.
Papers submitted can be between four and six pages in length, including references and appendix.

Submissions will be handled through   Easy Chair

Timeline will be:

  •     Submissions due: March 23, 2018 11:59PM US EDT

  •     Notification of outcome: April 13, 2018 11:59PM US EDT

  •     Camera-ready due: April 27, 2018 11:59PM US EDT

Workshop Program

Publication of all workshop papers: Proceedings of the Workshop on Testing Database Systems
All presentations for download
All videos from DBTest on youtube

8:00Opening    (introduction.pdf)
8:15Core DBMS Get Real: How Benchmarks Fail to Represent the Real WorldAdrian Vogelsgesang, Michael Haubenschild, Jan Finis, Alfons Kemper, Viktor Leis, Tobias Muehlbauer, Thomas Neumann and Manuel Thenpdf
Fair Benchmarking Considered Difficult: Common Pitfalls In Database Performance Testing Mark Raasveldt, Pedro Holanda, Tim Gubner and Hannes Mühleisenpdf
Finding the Pitfalls in Query Performance Martin Kersten, Panos Koutsourakis and Ying Zhangpdf
9:15Cloud and Big Data Snowtrail: Testing with Production Queries on a Cloud DatabaseJiaqi Yan, Qiuye Jin, Shrainik Jain, Stratis Viglas and Allison Leepdf
Performance of Containerized Database Management Systems Kim-Thomas Rehmann and Enno Folkertspdf
Adding Velocity to BigBenchTodor Ivanov, Patrick Bedué, Ahmad Ghazal and Roberto V. Zicaripdf
10:10Sponsor Talks Capturing bugs in extreme stress testing: Improving software quality in SAP HANA with UndoGreg Law, Undo; Stefan Bäuerle, SAP SEpdf
10:30Coffee Break
11:00KeynoteMachine Learning Testing and ProductivityManasi Joshi 
11:45Modern Hardware and New Scenarios Generating Evolving Property Graphs with Attribute-Aware Preferential AttachmentAmir Aghasadeghi and Julia Stoyanovichpdf
Make Larger Vector Register Sizes New ChallengesDirk Habich and Wolfgang Lehnerpdf
15:00 - 16:00Poster Session


Steering Committee:


  • Alexander Böhm (SAP SE, Germany)
  • Tilmann Rabl (TU Berlin, Germany)

Program Committee

  • Sihem Amer-Yahia (IMAG)
  • Artur Andrzejak (Uni Heidelberg)
  • Renata Borovica-Gajic (The University of Melbourne)
  • Geoffrey Fox (Indiana University)
  • Leo Giakoumakis (Snowflake)
  • Vasia Kalavri (ETH Zürich)
  • Eric Lo (CUHK)
  • Stefan Manegold (CWI)
  • John Poelmann (IBM)
  • Meikel Poess (Oracle)
  • Danica Porobic (Oracle)
  • Anna Queralt (Barcelona Supercomputing Center)
  • Norbert Ritter (Uni Hamburg)
  • Tiark Rompf (Purdue University)
  • Yogesh Simmhan (Indian Institute of Science (IISc), Bangalore)
  • Rekha Singal (TCS)
  • Markus Weimer (Microsoft)
  • Till Westmann (Couchbase)
  • Thomas Willhalm (Intel)
  • Jianfeng Zhan (Chinese Academy of Sciences)

Previous Editions of the Workshop