The Tensor Contraction Engine
The majority of software for scientific computations is written in the
lowlevel languages FORTRAN and C. The computational structure of
some of this software, however, has sufficient underlying structure
that it could benefit from specialpurpose software engineering tools
or domainspecific programming languages. E.g., electronic structure
calculations in quantum chemistry and in physics involve large
collections of tensor contractions (generalized matrix
multiplications). Currently, chemists spend weeks or months
manipulating formulas containing dozens or hundreds of terms with
Mathematica, handoptimizing the computation, and writing FORTRAN code
by hand. The computation can take on the order of 1 TFLOP week or
more and can require multiple TBs of storage.
We are developing a domainspecific language that allows chemists to
specify the computation in a highlevel Mathematicastyle language.
The compiler for this language, the Tensor Contraction Engine (TCE),
searches for an optimal implementation and generates FORTRAN code.
First, algebraic transformations are used to reduce the number
of operations. We then minimize the storage requirements to fit the
computation within the disk limits by fusing loops. We have
designed an algorithm that finds the optimal evaluation order if
intermediate arrays are allocated dynamically and are
working on combining loop fusion with dynamic memory allocation. If
the computation does not fit within the disk limits, recomputation
must be traded off for a reduction in storage requirements. If
the target machine is a multiprocessor machine, we optimize the
communication cost together with finding a fusion configuration for
minimizing storage. Finally, we minimize the data access times by
minimizing disktomemory and memorytocache traffic and
generate FORTRAN code. We have completed a first prototype of the TCE
and are working on implementing the communication minimization and
data access optimization algorithms. In future research, we will
extend this approach to handle common subexpressions, symmetric
matrices, and sparse matrices.
For an overview of the project, see our
Proceedings of the IEEE paper.
For more information about version 1.0 of the TCE (the "prototype"
TCE), please see our Getting and Using the
TCE page.
Investigators

P. Sadayappan
(PI), Dept. of Computer and Info. Sci., Ohio State

Gerald Baumgartner,
Dept. of Computer Sci., Louisiana State

David E. Bernholdt,
Oak Ridge National Lab.
 Robert J. Harrison,
Oak Ridge National Lab.

So Hirata,
Pacific Northwest National Lab.

Marcel Nooijen,
Dept. of Chemistry, University of Waterloo

Russell M. Pitzer,
Dept. of Chemistry, Ohio State

J. Ramanujam,
Dept. of Electrical and Computer Eng., Lousiana State
Senior Personnel
 Chitranjib Sur
Ohio State University
Current Students
Prior Contributors
 Alina Bibireata
 Daniel Cociorva
 Sandhya Krishnan
 Alexander Sibiriakov
Publications
 Hypergraph Partitioning for Automatic Memory Hierarchy
Management
S. Krishnamoorthy, U. Catalyurek, J. Nieplocha, and
P. Sadayappan.
Supercomputing'06, November 2006.
(SC 2006)
 Layout Transformation Support for the Disk Resident Arrays Framework
S. Krishnamoorthy, G. Baumgartner,
C. Lam, J. Nieplocha, and P. Sadayappan.
Journal of Supercomputing, vol: 36(2) pp:153170 May 2006.

Efficient Synthesis of OutofCore Algorithms
Using a Nonlinear Optimization Solver
S. Krishnan, S. Krishnamoorthy, G. Baumgartner, C. Lam,
J. Ramanujam, P. Sadayappan, V. Choppella.
Journal of Parallel and Distributed Computing,
vol:66(5) pp:659673 May 2006.

Automatic Code Generation for ManyBody Electronic Structure
Methods: The Tensor Contraction Engine
A. Auer, G. Baumgartner, D.E. Bernholdt, A. Bibireata,
V. Choppella, D. Cociorva, X. Gao, R.J. Harrison, A. Hartono,
S. Krishnamoorthy, S. Krishnan, C. Lam, Q. Lu, M. Nooijen,
R.M. Pitzer, J. Ramanujam, P. Sadayappan, A. Sibiryakov.
Molecular Physics, 2005.

Performance Modeling and Optimization of
Parallel OutofCore Tensor Contractions
X. Gao, S.K. Sahoo, Q. Lu, G. Baumgartner, C. Lam,
J. Ramanujam, P. Sadayappan.
Proceedings of the ACM SIGPLAN 2005
Symposium on Principles and Practice of Parallel Programming,
Chicago, Illinois, 1517 June 2005.

Automated Operation Minimization of Tensor Contraction
Expressions in Electronic Structure Calculations
A. Hartono, A. Sibiryakov, M. Nooijen, G. Baumgartner,
D.E. Bernholdt, S. Hirata, C. Lam, R.M. Pitzer, J. Ramanujam,
P. Sadayappan,
Proceedings of the International Conference on
Computational Science 2005 (ICCS 2005),
Atlanta, Georgia, 2225 May 2005.

Synthesis of HighPerformance Parallel Programs for a Class
of Ab Initio Quantum Chemistry Models
G. Baumgartner, A. Auer, D.E. Bernholdt, A. Bibireata, V. Choppella,
D. Cociorva, X. Gao, R.J. Harrison, S. Hirata, S. Krishnamoorthy,
S. Krishnan, C. Lam, Q. Lu, M. Nooijen, R.M. Pitzer, J. Ramanujam,
P. Sadayappan, A. Sibiryakov.
Proceedings of the IEEE, Vol. 93, No. 2,
February 2005, pp. 276292.

Efficient Layout Transformation Support for Diskbased
Multidimensional Arrays
S. Krishnamoorthy, G. Baumgartner, C. Lam, J. Nieplocha,
P. Sadayappan.
In L. Bougé, V.K. Prasanna (eds.),
Proceedings of the 11th Annual International Conference on
HighPerformance Computing (HiPC '04),
Bangalore, India, 1922 December 2004.
In Lecture Notes in Computer Science, Vol. 3296, SpringerVerlag,
pp. 386398.

Layout Transformation Support for the Disk Resident Arrays
Framework.
S. Krishnamoorthy, G. Baumgartner, C. Lam, J. Nieplocha,
P. Sadayappan.
Proceedings of the
Los Alamos Computer Science Initiative Symposium.
Santa Fe, New Mexico. 1214 October 2004.

Empirical PerformanceModel Driven Data Layout Optimization.
Q. Lu, X. Gao, S. Krishnamoorthy, G. Baumgartner, J. Ramanujam,
P. Sadayappan.
In R. Eigenmann, Z. Li, S. Midkiff (eds.),
Languages and Compilers for Parallel Computing,
West Lafayette, Indiana, 2225 September 2004.
 A HighLevel Approach to Synthesis of HighPerformance Codes
for Quantum Chemistry: The Tensor Contraction Engine
G. Baumgartner, D.E. Bernholdt, V. Choppella, J. Ramanujam,
P. Sadayappan.
In Proceedings of the 11th Workshop
on Compilers for Parallel Computers (CPC 2004),
Chiemsee, Germany, 79 July 2004, pp. 281290.

Efficient Synthesis of OutofCore Algorithms Using a
Nonlinear Optimization Solver
S. Krishnan, S. Krishnamoorthy, G. Baumgartner, C. Lam,
J. Ramanujam, P. Sadayappan, V. Choppella.
Proceedings of the International Parallel
and Distributed Processing Symposium (IPDPS '04),
Santa Fe, New Mexico, 2630 April 2004, Abstract p. 34b, 10 pages.
Best paper award.

Efficient Parallel OutofCore Matrix Transposition
S. Krishnamoorthy, G. Baumgartner, D. Cociorva, C. Lam,
P. Sadayappan.
International Journal on High Performance Computing and
Networking, 2004.
A previous version
of this paper has appeared
in the Proceedings of the IEEE International Conference on
Cluster Computing (Cluster '03), Hong Kong, China,
14 December 2003, IEEE Computer Society Press, pp. 300307.

Data Locality Optimization for Synthesis of Efficient
OutofCore Algorithms
S. Krishnan, S. Krishnamoorthy, G. Baumgartner, D. Cociorva,
C. Lam, P. Sadayappan, J. Ramanujam, D.E. Bernholdt, V. Choppella.
In Proceedings of the International Conference on
HighPerformance Computing (HiPC '03),
Hyderabad, India, 1720 December 2003.
In Lecture Notes in Computer Science, Vol. 2913, SpringerVerlag,
pp. 406417. Best paper award.

Efficient Parallel OutofCore Matrix Transposition
S. Krishnamoorthy, G. Baumgartner, D. Cociorva, C. Lam,
P. Sadayappan.
In Proceedings of the IEEE International Conference on
Cluster Computing (Cluster '03), Hong Kong, China,
14 December 2003, IEEE Computer Society Press, pp. 300307.
An
extended version of this paper will appears in
International Journal on High Performance Computing and
Networking, 2004.

MemoryConstrained Data Locality Optimization for
Tensor Contractions
A. Bibireata, S. Krishnan, G. Baumgartner, D. Cociorva, C. Lam,
P. Sadayappan, J. Ramanujam, D.E. Bernholdt, V. Choppella.
In L. Rauchwerger (ed.),
Proceedings of the 16th International Workshop on
Languages and Compilers for Parallel Computing (LCPC '03),
College Station, Texas, 24 October 2003, SpringerVerlag,
Lecture Notes in Computer Science, Vol. 2958, 2004, pp. 93108.

On Efficient OutofCore Matrix Transposition
S. Krishnamoorthy, G. Baumgartner, D. Cociorva, C. Lam,
P. Sadayappan.
Technical Report OSUCISRC9/03TR52, Dept. of Computer and
Information Science, The Ohio State University, September 2003.

Global Communication Optimization for Tensor Contraction
Expressions under Memory Constraints
D. Cociorva, X. Gao, S. Krishnan, G. Baumgartner, C. Lam,
P. Sadayappan, J. Ramanujam.
In Proceedings of the International Parallel and
Distributed Processing Symposium (IPDPS '03),
Nice, France, 2226 April 2003, Abstract p. 37b, 8 pages.
 CompileTime Optimizations for Tensor Contraction Expressions
G. Baumgartner, D. Cociorva, C. Lam, P. Sadayappan, J. Ramanujam.
In Proceedings of Compilers for Parallel Computers
(CPC '03), Amsterdam, The Netherlands, 810 January, 2003.

A HighLevel Approach to Synthesis of HighPerformance Codes for
Quantum Chemistry
G. Baumgartner, D.E. Bernholdt, D. Cociorva, R.J. Harrison, S. Hirata,
C. Lam, M. Nooijen, R.M. Pitzer, J. Ramanujam, P. Sadayappan.
In Proceedings of Supercomputing 2002,
Baltimore, Maryland, 1622 November 2002.
IEEE Computer Society Press, Abstract p. 5, 10 pages.

MemoryConstrained Communication Minimization for a Class of
Array Computations
D. Cociorva, G. Baumgartner, C. Lam, P. Sadayappan, J. Ramanujam.
To appear in B. Pugh, C. Tseng (eds.),
Proceedings of the 15th International Workshop
on Languages and Compilers for Parallel Computing (LCPC '02),
College Park, Maryland, 2527 July 2002.

Automatic Synthesis of HighPerformance Codes for Quantum
Chemistry Applications
G. Baumgartner, D.E. Bernholdt, D. Cociorva, R.J. Harrison,
C. Lam, M. Nooijen, J. Ramanujam, P. Sadayappan.
To appear in Proceedings of the Workshop on Performance
Optimization for HighLevel Languages and Libraries
(POHLL02), New York, New York, 22 June 2002.

MemoryOptimal Evaluation of Expression Trees Involving
Large Objects
C. Lam, T. Rauber, G. Baumgartner, D. Cociorva, P. Sadayappan.
Submitted to ACM Transactions on Programming Languages
and Systems (TOPLAS), 2002. Currently under revision.

SpaceTime TradeOff Optimization for a Class of Electronic
Structure Calculations
D. Cociorva, G. Baumgartner, C. Lam, P. Sadayappan,
J. Ramanujam, M. Nooijen, D.E. Bernholdt, R.J. Harrison.
In Proceedings of the ACM SIGPLAN 2002 Conference on
Programming Language Design and Implementation (PLDI '02),
Berlin, Germany, 1719 June 2002, pp. 177186.

A Performance Optimization Framework for Compilation of Tensor
Contraction Expressions into Parallel Programs.
G. Baumgartner, D.E. Bernholdt, D. Cociorva, R.J. Harrison,
C. Lam, M. Nooijen, J. Ramanujam, P. Sadayappan.
7th International Workshop on HighLevel Parallel
Programming Models and Supportive Environments (HIPS '02),
In Proceedings of the International Parallel and Distributed
Processing Symposium (IPDPS '02),
Fort Lauderdale, Florida, 15 April 2002,
IEEE Computer Society, pp. 106114.

Towards Automatic Synthesis of HighPerformance Codes for
Electronic Structure Calculations: Data Locality Optimization
D. Cociorva, J. Wilkins, G. Baumgartner, P. Sadayappan, J. Ramanujam,
M. Nooijen, D.E. Bernholdt, R.J. Harrison.
In Proceedings of the International Conference on
HighPerformance Computing (HiPC '01),
Hyderabad, India, 1721 December 2001, SpringerVerlag, Lecture Nodes
in Computer Science, Vol. 2228, pp. 237248.

Loop Optimizations for a Class of MemoryConstrained
Computations
D. Cociorva, J. Wilkins, C. Lam, G. Baumgartner, P. Sadayappan,
J. Ramanujam.
In Proceedings of the 15th ACM International Conference on
Supercomputing (ICS '01),
Sorrento, Italy, 1621 June 2001, pp. 103113.

MemoryOptimal Evaluation of Expression Trees Involving Large
Objects
C. Lam, D. Cociorva, G. Baumgartner, P. Sadayappan.
In Proceedings of the 1999 International Conference on High
Performance Computing (HiPC '99), Calcutta, India,
1720 December 1999, IEEE Computer Society, SpringerVerlag,
Lecture Notes in Computer Science, Vol. 1745, pp. 103110.

Optimization of Memory Usage and Communication Requirements for
a Class of Loops Implementing MultiDimensional Integrals.
C. Lam, D. Cociorva, G. Baumgartner, P. Sadayappan,
In J. Ferrante, L. Carter (eds.),
Proceedings of the 12th International Workshop on
Languages and Compilers for Parallel Computing (LCPC '99),
La Jolla, California, 46 August 1999, SpringerVerlag, Lecture Notes
in Computer Science, Vol. 1863, pp. 350364.
Software
Funding
Gerald Baumgartner
Last modified: Tue Mar 15 23:31:21 CST 2005