Posted Dec 1, 1995

## Rules for the Computer Simplification of the Formulas in Operator Theory and Linear Systems

### By J. W. Helton and J. J. Wavrik

This article formulates and treats questions in operator theory arising from computer simplification of formulas commonly found in the study of operator models. Operator model theory originated with Moshe Livsic and subsequently became one of the main branches of operator theory. In studying a particular operator model polynomials in certain expressions occur repeatedly. This makes it a natural area for exploring computer algebra simplification.

The purpose of a simplification theory is to provide a means for replacing complex expressions by expressions which are ``simpler'' in some sense. The main task is to obtain a list of rules each of which replaces a ``complicated'' monomial which occurs in an expression by a sum of ``simpler'' monomials. To simplify such an expression one applies the rules to the expression until no further reduction is possible. The result is called an N-Form (normal form) for the original expression. The reduction of an expression to an N-Form can be easily implemented on a computer. This article provides a collection of reduction rules for expressions which arise in the Nagy-Foias operator model. Our simplifying rules were obtained by applying an algorithm for computing a Gröbner basis for an ideal in a polynomial ring. It is applied to the ideal generated by a set of fundamental relations which obviously hold for NF calculations. We conjecture that all appropriate relations are in this ideal. The algorithm produces an infinite set of rules in the NF case. The traditional operator theorist's functional calculus is used to produce a nice formulation of the rules as a finite set.

If a set of generators is a Gröbner Basis, then the reduction to an N-form has very nice properties; the N-form is independent of the order in which rules are applied and equality of N-forms can be used as a test of equivalence of expressions mod the ideal. We have established that our rules form a Gröbner Basis for several situations. A proof outline and computer tests provide strong evidence that this is true for the NF case as well.

The results here have potential applications to engineering systems theory since this algebraic structure occurs in formulas arising in H control. Also since the entries of a unitary 2x2 block matrix have algebraic structure similar to the NF model, this applies to `all pass' functions from engineering.