The Filter Of Observation

The correspondence between continuous and discrete transfer functions 
in signal filtering involves many of the same issues that arise in the 
consideration of measurement in quantum mechanics.

Let x(t) signify a real physical variable regarded as a continuous 
function of time.  We might attempt to construct a model dealing 
directly with x(t), but since any transfer of information must itself
constitute a physical effect (i.e., we cannot observe x without 
physically interacting with it) we should base our model on a dynamic 
coupling to x(t) rather than on x(t) itself.  A typical continuous-time
dynamic coupling (i.e., filter) can be expressed schematically as a 
transfer function

           _________________________
          |  a0 + a1 s +...+ an s^n |
 x(t) ____|  ______________________ |____ y(t)
          |  b0 + b1 s +...+ bn s^n |
          |_________________________|

where 's' denotes the differential operator.  Here x(t) is the subject
physical variable and y(t) is our observation of x(t) achieved via the
coupling.  This transfer function simply signifies that x(t) and y(t) 
are related according to the ordinary differential

        n      d^j x         n      d^j y
      SUM  aj -------  =   SUM  bj ------              (1)
       j=0     d t^j        j=0     d t^j

It's worth noting that this coupling is symmetrical; neither x(t) nor 
y(t) must necessarily be regarded as the independent or dependent 
variable.  The transfer function simply establishes a coupling between 
the two; y(t) is our observation of the system variable x(t), and 
conversely x(t) is the system's "observation" of our variable y(t).

Of course, this coupling by itself does not uniquely determine either 
x(t) or y(t).  Each of these variables is subject to its own system 
constraints.  By examining y(t) we hope to discern the constraints on 
x(t), from which we will infer something about the system of which x 
is a part.  At the same time the constraints we impose on y(t) on our 
side of the coupling have an influence on x(t) and the system being
observed.

The situation becomes more interesting when we attempt to translate 
such a coupling into the discrete-time domain (as, for example, when 
we model a continuous filter on a digital computer).  In this case we 
can deal explicitly only with the values of the variables at discrete 
time intervals, i.e., we have only the values x(kT) and y(kT) where
k=..-1,0,1,2.. and T is a constant time increment.

How do we translate the continuous transfer function into an equivalent
coupling between the discrete-time values of x and y?  There is no 
single discrete-time formula that will match the continuous relation 
for all possible signals x(t) and y(t) (because the discrete-time model 
has no knowledge of the behavior of the signals at frequencies greater 
than 2pi/T.)  However, there are two specific translations that are, 
in different respects, optimum.  One of these can be identified with 
"observed interactions" while the other can be identified with 
"unobserved interactions".

Suppose we intend to make a measurement of the variable x(t) associated
with a particular system.  For this purpose we design an interaction
between x(t) and our local variable y(t) such that y(t) is as free of
constraints (on our side of the coupling) as possible.  In this context 
we can essentially treat x(t) as an independent variable and y(t) as 
the dependent variable (i.e., entirely dependent on x(t)).  Now, given 
a sequence of discrete values x(0), x(T), x(2T),...x(nT) we still need 
to assume something about the form of the continuous variable x(t) in 
order to solve equation (1) for y.  Of all the possible continuous
functions the "most probable" is the unique nth degree polynomial 
that passes through the given values of x(kT).  On that basis we can 
define the unique discrete-time recurrence relation corresponding to 
(1) with matched homogeneous response for y(t).

In contrast, consider an interaction in which x(t) and y(t) are 
symmetrical with respect to our state of knowledge, i.e., neither of 
them is regarded as 'given'.  In this context the optimum discrete-
time recurrence is given by matching the homogeneous response of (1) 
"in both directions", i.e., for both x(t) and y(t).  The resulting 
discrete-time recurrence is symmetrical and reversible.

Letting X and Y denote the column vectors with the components x(kT) and
y(kT) respectively (for k=0,1,..,n) the general form of the discrete-
time recurrence is  S_a Y = GX  where S_a and G are constant row 
vectors.  The components of S_a are the elementary symmetric functions 
of the exponentials of the roots of the characteristic equation of the
right side of (1).  Similarly we define the row vector S_b in terms of 
the characteristic roots of the left side of (1).

The components of G depend on which of the two contexts is assumed.  
For a "measurement coupling" when y(t) is treated as a purely 
dependent variable we have

           G = (S_a)(M)(A^-1)(B)(M^-1)            (2)

where A, B, and M are square matricies defined by

              / (j!/k!)a_(j-k)   if j >= k
     A_k,j = ( 
              \       0          if j < k


              / (j!/k!)b_(j-k)   if j >= k
     B_k,j = ( 
              \       0          if j < k


              /   (kT)^j      if j >= k
     M_k,j = ( 
              \     0         if j < k


On the other hand, if we treat x(t) and y(t) symmetrically (i.e., a
non-measurement coupling) we have

                  b_0   SUM(S_a)
             G = -----  --------  S_b                (3)
                  a_0   SUM(S_b)

The G vectors given by (2) and (3) converge to each other in the limit 
as T goes to zero.  Combining these equations, we find that the vector 
given by
              a_0
           -------- (S_a)(M)(A^-1)
           SUM(S_a)

is invariant, i.e., it approaches the same vector as T goes to zero,
regardless of the components of A.  The most significant non-zero term 
of the components of this N+1 dimensional vector are of the form

              T        T^2            T^N
    c0,   c1 --- ,  c2 --- , ... , cN ---
              2!        3!             N!

where the coefficients ck are as shown below

                       k
    N     0    1    2    3     4     5
   ---  -----------------------------------
    1     1    1
    2     1    2    7
    3     1    3   15   108
    4     1    4   26   240  2916
    5     1    5   40   450  6620  121500

These coefficients can be generated using Eulerian numbers, and are 
closely related to the generalized Bernoulli numbers.

What I find most interesting about these two forms of filters is 
that when we impose a "direction" on the transfer of information by
determining one of the two sides of the equation, the resulting 
"most probable" transfer is irreversible, whereas if we allow the
interaction to exist "unobserved" the most probable transfer is 
perfectly time-symmetric and reversible.  This seemingly paradoxical 
situation arises only in the discrete-time case when the elementary
increment of time T is non-zero.