# Readings 6 – Inverse Kinematics

by on February 8, 2011 · 19 comments

For this class, I want you to get a sense of the basic approaches for inverse kinematics. What the challenges are, what the general categories of methods are, and why you might choose among them.

Everyone should:

• Read one of the papers (either Buss or Maciejewski) about numerical approaches. Buss is much more “here’s how to do it”, but Maciejewski is the classic presentation of the core issues. Baerlocher and Boulic have a method with an interesting twist. Don’t worry about the details of the methods (if you go down this road, you might want to apply some “real” optimziation techniques).
• Read a paper that applies an analytic technique (either the Kovar footskate paper, or the recent Kallman paper).
• Read a paper about example based methods – preferably one of the newer ones (Xu or Wei).

Hopefully, different people will read different things, so that as a class we’ll collectively learn about them all. In the future, I might need to seed this, but for now, we’ll hope for randomness).

Yes, you need to read 3 different papers. But there’s no reading for Wednesday, and you don’t necessarily need to read everything in gory detail – just try to get the main ideas.

Then, in a comment to this page (posted at least 1 hour before class on Friday) please say:

• Your thoughts to the highlights of these papers (what are the key ideas, suprising results, … -especially considering that others might not have read the same papers as you).
• Your thoughts on where the different approaches may be appropriate. Especially considering that some of the newer methods are much more “expensive” (in various ways) than other approaches.

Note: in class, we might spend more time discussing papers that I am not having you read (about how to apply these methods to whole motions).

##### Inverse Kinematics

Again, this is such a basic topic that I am surprised that I don’t have a good tutorial.

Numerical approaches: the first is a seminal paper is one that really starts to get at core issues with numerical methods. The second is a tutorial of methods applied in practice. The third is an interesting technique with useful features.

Analytic Approaches:

Inverse Kinematics by Example: A newer way to solve IK problems is to use a database of examples. This was pioneered by Rose et al, had complicated machine learning math thrown at it, and most recently huge databases.

Michael Correll February 10, 2011 at 5:50 pm

I read the Wu, Maciejewski, and the Tolani et al. papers.

It was surprising just how bad the early results were (the Tolani paper was especially bad). I think the lesson from the early numerical paper I read was “human motion IK is a hard problem, even if you are doing it in a reasonably sophisticated way.”

The Wu paper had some good results, but that was to be expected. “Get a bunch of data and throw some ML at it” pretty much always produces good results, for certain values of “good.” Since the human IK problem seems so difficult, maybe the Wu paper is the right way to think about it, especially as computational power has quickly gotten much cheaper than hiring animation talent to clean data.

Jim Hill February 10, 2011 at 8:04 pm

I started reading Maciejewski and decided the math was to dense and decided to try the Buss paper. Turns out that one was pretty dense to but I plowed through it possibly not understanding a lot of it. The title really describes the paper well, it’s basically an intro to different methods of solving the IK problem.

The second paper I chose was “Footskate Cleanup for Motion Capture Editing.” For those of you who didn’t read the paper, footskate is what happens when raw motion capture data is turned into a skeleton. What happens is that foot plant data is messy and the fool ends up looking like it’s sliding along the ground. This paper proposed a solution to the problem and it looks like it works pretty well. The paper states that it’s an Analytic solution which I’m guessing is more like a procedural solution than a full blown non-linear system solving numeric solution. This is definitely relevant in film and games as it really helps the animation look real and physically plausible.

The final paper I read was “Artist-Directed Inverse-Kinematics Using Radial Basis Function Interpolation”. This sort of falls into the category of “Throw data at the problem.” But it seems like most of the data is coming from actual animators. Basically a few examples are provided and in each example, verbs and adverbs are defined and the software can interpolate between these. Example adverbs might be emotional, like happy or sad, but the paper made it sound like they thought of adverbs in more of a geometric sense, like positional and rotational. Their paper describes a method for solving the IK problem using radial basis function interpolation. I’m not sure what a radial basis function is, but apparently they can create motion at thousands of frames a second which makes this a great candidate for use in video games.

sghosh February 10, 2011 at 9:45 pm

The first paper that I read was the Maciejewski paper. This paper is quite old – from 1990 and it is interesting to note that so much work on IK was going on then. This paper deals with how to handle certain ill-conditioned cases which arise while solving IK problems. The problem arises because of singular positions of the skeleton (say when the arm is totally stretched out – and the joints in the arm become linearly dependant). What happens as a result is that one of the axes of motion becomes redundant and there is a discontinuous transition of one of the parameters (this is similar to boundary discontinuities in certain parametrization methods for rotation). To remove such conditions and to make it ‘well-conditioned’ a damping factor is introduced into the formulation and this factor restores continuity near singularities. The whole idea is to move a skeleton from one point to the other with minimum joint motion and least error.

The second paper I read was the Footskate paper. As mentioned by Jim, the foot seems to ‘skate’ or slip along the ground and this results in unrealistic motion. To solve this problem, the authors have used an unusual but very interesting idea – to make the skeleton non-rigid i.e. bones can change their lengths without significant perceptual disturbance. This method can also modify the position of the root and orientation of non-end effector joints. The algorithm also smoothly merges these changes with the neighbouring frames to prevent any weired motion across frames. All these result in much believable footplants.

The third paper I read was the Wu paper. The main challenge that the paper tries to solve is how to generate poses from a large using a large database of motion and they come up with the NAT-IK algorithm. Prior to this, another algorithm existed called Style-IK and was quite successful. Style-IK however has very limited training capacity and as a result it fails to deliver good results when the desired pose is much different from what was there in the small training set. NAT-IK does not suffer from this problem and can read in huge databases and uses recursive kd-clustering to group pose representation frames and uses some complicated approximations along the way. The algorithm thus allows creation of more natural ‘intermediate’ poses and performs pretty well compared to other methods.

sghosh February 10, 2011 at 10:04 pm

Ahh I forgot the third part of the question and I could not find any way to edit my comment…

I believe both the Wu and the Footskate paper are extremely relevant for games and animated movies. Creating human like believable poses is a high priority for any animator. Given that the NAT-IK algorithm reduces the time to train data by 2 order of magnitude and the time to synthesize a pose by a magnitude (the only step that takes time is the one time initial data selection) this method is desirable.

adrm February 10, 2011 at 10:33 pm

I read the Maciejewski, Kallmann, and Wei papers.

The Maciejewski is interesting from a historic perspective, as it is 21 years old. Comparing this to newer stuff, it seems like a lot fancier stuff is going on in the newer methods, though its hard to compare the difference in the actual approaches from the paper. Most of the paper deals with how to prevent singularities from messing up the optimization.

The second paper deals with this in a completely different manner, by throwing away all(ok most) numerical stuff and manually coming up with a parametrization for valid human poses. A search over this space is then finds and acceptable pose that meets the constraints. From the examples, this does not seem to work as well, since the poses tend to look unnatural.

The last paper literally throws millions of poses to the problem. The results here are excellent. Even more impressive, is the fact that the search and constraint solving can be solved at interactive rates, allowing even a novice user to generate good poses. While this has the downside of needing lots of data, it is obvious that it exists out there already.

With these approaches in mind, I would say that if the time and memory constraints permit it, example based methods are probably the best option. When memory usage is a concern, such as in video games, it seems that analytical techniques would be better. Finally, numerical, optimization based approaches might be better for getting good poses when there is not enough data to reconstruct from examples.

Leslie February 10, 2011 at 10:38 pm

I read the Maciejewski, Kallmann, and Wei papers.

All three papers were very mathematically dense. The Maciejewski paper first defined what it means for the equations of motion of an articulated figure to be ill conditioned, namely that any calculation errors will be magnified dramatically (like if the arm were in a position where a small change in the location of the hand will cause the arm to rotate through its full range of motion). It then explains how incorporating a damping factor into these equations will make them well-conditioned. The Kallmann paper describes a method for quickly and accurately animating reaching motions. The user specifies where the figure should reach and provides some reference body poses, and the method interpolates between a selected subset of these poses to create a natural-looking reaching motion. The Wei paper describes a related technique for creating arbitrary poses using a combination of user input (either through direct manipulation or sketching) and a huge database of mocap data.

The Kallman and Wei papers both mention that their methods have applications in real-time character control, such as video games. Also, both are highly automated and seem to be meant for the novice user. The method in Wei’s paper will become more and more relevant and useful as motion capture databases grow.

csv February 10, 2011 at 11:00 pm

Paper 1: Samuel Buss.
Introduction to Inverse Kinematics with Jacobian Transpose, Pseudoinverse and Damped Least Squares methods

This paper is remarkable for its findings and comparing the results with experiments. It shattered my strongly held concepts that Penrose Pseudoinverse method and SVD are ultimate in least square solvers.

I could never understand (geometrically) he NULLSpace methods and it seems that
they are also used in IK.

Although in this paper,the authors call is “Dumped Least Square” method, the
technique is widely known as “Tikhonov Regularization” and is well know in
mathematics.
**********************************************************************
Paper2 : Marcelo Kallmann. Analytical inverse kinematics with body posture control. Computer Animation and Virtual Worlds, 19 (2) May 2008.

This paper gives some mixed feeling to a beginner in the field of animation

mathematical equations to model the system. ( Except the Quaternion, all things
are learned in high school).

The most surprising element in this paper was the result showing 12K
triangulation ( with incremental algorithm ) is less than 10 milliseconds. I
am not quite sure about the rationale behind using Delunay method.

*********************************************************************
Paper 3: Intuitive Interactive Human Character Posing with Millions of Example Poses
`
The title is attractive and idea is simple, but the mathematical model using MAP produces results that have risks of ambiguities.

I am curious to know how the information from the millions poses were extracted and stored. I am not even sure if millions of poses are needed. My belief is that human actions are extremely limited and in that million database, authors didn’t mention how many redundant poses were present.

The second element of surprise was the vector of 42 dimension.

I am curious to know whether database of millions poses increases or decreases
ambiguity to the end users.

These types of papers have characteristics of non-reproducibility and therefore
whatever said in the paper must be taken with the pinch of salt.

csv February 10, 2011 at 11:15 pm

Even I missed the question #3 and here is my understanding.

The first paper by Buss could be used for high quality animation software where the results are more important than real time behaviour. Whereas, Kallman work is suitable for low end animation systems or games where people get more focused on the context rather than contents of the animation. In such systems, having a fast and approximate model is more than sufficient.

I am quite skeptical of usefulness and effectiveness of “Millions of Example” paper. I think rules ( small number ) based systems could be better to mimic the humans varied actions ( a la Wolfram )

xlzhang February 10, 2011 at 11:52 pm

I read the Buss, Kovar, and Grochow papers.

For the Buss paper, I found the paper moderately dense, but it was a comprehensive overview which was necessary for understanding the basic approaches which are out there. However, the next two papers I read were more interesting and unique because they seemed to simplify the problem of IK without sacrificing too much in terms of accuracy and results. The Kovar paper was interesting because it actually suggested stretching as a solution, which I had not seen before, and it is innovative in its unorthodoxy.

The Style-Based IK paper was also interesting because of synthesizing poses, and methods to interpolate poses based on this method, which is something we talked about in class and may need to do for our projects. Also, it is just interesting because more than any other paper, it seems to try to abstract on data and move away from oversized data sets.

sgallege February 11, 2011 at 12:22 am

* Dealing with the Ill-Conditioned Equations of Motion for Articulated Figures , Anthony Maciejewski
The paper identifies two main problems that arise from motion data: numerical instability and ill conditioning and focuses methods of dealing with ill conditioning. The authors explain how Singular value analysis and pseudo inverse solutions fail to solve the problem by them self due to singularities. By considering both criteria simultaneously come up with better solution that doesn’t have discontinuities. The new method is called damped least squares solution.

* Footskate Cleanup for Motion Capture Editing by Lucas Kovar John Schreiner Michael Gleicher, SCA 2002.
This paper proposes a way to clean up a condition known as footskate (an artifact caused by feet moving when they are supposed to planted). The key idea of the paper is to not worry about preserving the length of the foot skeletons. By making the length of the flexible and considering the foot movement of the neighboring frames they manage to get a more realistic walking motion. I am surprised how much work had to be put in to solve a seeming minor issue.

*Natural Character Posing from a Large Motion Database. Wu, X; Tournier, M; IEEE CG&A November 2009.
This paper proposes a method of generating new natural looking poses from a large motion database. They have used kd-clustering to learn the priors from a large data base taking advantage of the fact that the poses are highly redundant. The algorithm is how ever limited to generating poses and doesn’t provide continuous motion and doesn’t work in real time.

danieljc February 11, 2011 at 12:28 am

I read through the Maiejewski, Wu, and Kallmann papers.

The Maciejewski paper talked about some of the problems encountered when modeling simulations. It mentioned how these problems are often caused by the model itself rather than the equations. The Kallmann paper went into more inverse kinematics and how this can be used for animating specific contact or collisions more accurately. Finally, the Wu paper discussed methods for taking a subset of data from a motion database and using this creating more realistic inverse-kinematics. In general, it seems like a problem with inverse kinematics is that it can result in unrealistic motion between positions.

As for applications, it doesn’t seem like the ideas from the Wu paper would be as good for an interactive situation, compared to the other two papers. First, it needs to be trained on the data and then it still can take multiple seconds to generate a pose even after the training. Overall, inverse kinematics seems like something that would be useful, at least in some capacity, for most animation tasks.

Reid February 11, 2011 at 12:34 am

Dealing with the Ill-Conditioned Equations of Motion for Articulated Figures

This paper discusses the nature of the equations used for motion in articulated figures. In particular, the equations are ill-conditioned, meaning that no matter what you do, there is the potential that small perturbations in the input can cause large changes in the output. One tactic is to generate solutions using the pseudoinverse can result in discontinuous transitions. The preferred solution is using damped least-squares formulation which can be arbitrarily well-conditioned with the correct damping setting.

From what I gathered, it sounds like the psuedoinverse calculation is faster than the damped least-squares formulation. If that is the case, then pseudoinverse may be more appropriate for applications that demand speeds, such as games, while damped least-squares would be better for creating movies or other offline animation.

Footskate Cleanup for Motion Capture Editing

This paper proposes a post-processing method of removing footskate without introducing artifacts to the motion. This is achieved by making small changes to bone lengths in order to meet constraints without generating large changes in joint rotation. Since it is a post process, it doesn’t interfere with other algorithms, and allows other motion processing algorithms to introduce some footskate, knowing it will be removed in the post processing step.

Since the algorithm is post processing, it might not be suitable for real time application that demand a fast response. The paper suggested using it for characters in the backgrounds of games or crowed simulations, situations where the movements aren’t under intense scrutiny. Other methods, if faster, may be more applicable to conditions where speed is desired.

Natural Character Posing from a Large Motion Database

This paper proposes a method (nat-ik) for learning a prior for pose from a large motion database while maintaining interactive processing speed. This method is to be general over a human reachable space, unlike style-ik. The method deals with large databases by clustering poses and selecting representative frames from each cluster. FITC was used to approximate the Gaussian Process model, chosen for training speed, prediction speed, and error rate.

The impression I got was that that Nat-IK was a more general solution than style-ik, meaning style-ik might be more appropriate when a subset of motions is desired in higher accuracy.

Aaron Bartholomew February 11, 2011 at 1:21 am

-Dealing with Ill-Conditioned Equations of Motion for Articulated Figures:
Maciejewski presents a numerical approach for solving the joint inputs when the constraints are ill-formed. This situation occurs when the singular values of the Jacobian matrix (scaling factors based upon the size of the axes of the elliptical space in which the end effector can move) approach zero; an example is when the joint chain is fully stretched out. He points out how pseudoinverse solutions, which are usually used to compensate for these singularities, lead to discontinuous transitions between configurations and can ‘wreak havoc on numerical integration routines’ due to sharp spikes/oscillations in derivatives. His solution to this is called “damped least squares” which essentially reduces the size of the solution space to physically realistic joint velocities. Considering that this was my first exposure to the inner-workings of IK, I was really surprised to not only see the complexity of the math but also how unstable it seems to be. The numerical approach seems fairly susceptible to break down and must introduce a lot of error; although I can’t personally attest to the quality of the damped least squares method, it seems to be more of a way to conceal the problem than solve it.

-Real-Time IK for Anthropomorphic Limbs:
This paper did a great job of defining the issues that make solving IK in computer graphics difficult and the criteria for which IK methods should be evaluated. The authors determine that the main issues with CG IK are:
1)The amount of constraints to account for which can be simultaneous and conflicting.
2) Large # of degrees of freedom
3)Many joints are coupled (not independent and may move simultaneously)
4)Large amount of error
They go on to introduce an analytic/hybrid approach to IK and compare it with traditional numerical methods. Their method is faster and complete (in terms of solutions it accounts for) and far less unstable (it produces repeatable results which doesn’t happen numerically since the methods rely on initial guesses). Finally, they also suggest that the quality of IK methods should be based upon: efficiency, reliability, completeness, stability, and generality. Without a doubt, this paper provided the solution that Maciejewski’s circumvented; the method just seems to crush the purely numerical approach.

-Style-Based IK
Presents a method for using a probability distribution-based solution space for determining poses that follow a style learned from example data. The authors argue that most IK systems fail to keep solutions within a natural-looking space, making it so animators have to specify more constraints than necessary. Their method reduces this unnatural space by tending to choose solutions that closely resemble the training poses. I definitely see potential in this method; from my experience, rigging characters can be a pain in the ass, so the added constraint automation would alleviate some of this pain. Although, I do find the necessity for examples to be pretty limiting. Can you easily acquire the data to teach the intended style? And will the intended style always come through correctly? If the data is difficult to acquire and communicate effectively, then maybe it’s less trouble to just manually specify the constraints.

Although I’ve been informed that the numerical approaches are better suited for ill-formed constraints, it really seems like the analytic/hybrid approach is so much better. Maybe it’s the age of the Maciejewski paper, but the efficiency and repeatability of the analytic/hybrid would be so much better for any real-time application. Between the Numerical/Analytic approaches and the data approaches, it seems like it really just comes down to control versus speed. Obviously the data-driven approach is going to produce quality results quickly during production, but its diversity of movement is limited to the data you provide. On the other hand, although far more costly (in terms of time), numerical/analytic offers more control and consequently, more diversity. Using a database of millions of poses seems like it would be costly, redundant, and probably not dynamically usable in real-time, but it is needed in order to get diversity in pose solutions.

Thus, in terms of gaming, I can see data-driven examples being used for rapid development of animation sets for minor characters, while numerical/analytic being used for in-depth development of animation sets for main characters. Also, numerical/analytic would more likely be used in real-time computation due to efficiency and space (since a database of poses wouldn’t have to be loaded into memory or read from elsewhere).

Nathan Mitchell February 11, 2011 at 1:44 am

I read the Maciejewski, Kallmann, and Wei papers.

The basic outline of the Maciejewski paper follows
the variations of using Jacobian based numerical
solutions to IK. The author explains how
ill-conditioned problems, primarily due to cases
where the Jacobian matrix is singular, can be
solved with minimal error by using the damped
least-squares method. He then explains how this
method can be applied to general dynamics problems
with good results.

I found the second paper harder to follow, but I
believe the basic idea was to develop closed form solutions to
IK problems in contrast to the numerical solutions
presented in the first paper. In addition, the
paper demonstrates a method for producing a
bipedal motion that maintains a proper posture
when the arms are given constraints. In some
respects, the use of a set of precomputed posture
‘stereotypes’ is similar to the third paper,
except the third paper pulls from a far larger
database of natural motion.

The third paper, by Wei, makes use of machine
learning techniques to distill a large database of
motions into a model that can be used to
intelligently fill in the gaps left by user
constraints. The author describes the purpose as a
system that allows novice animators to present a rough
sketch or design of a pose and the model
constructs what would be the most natural looking
pose to fit with the constraints. This paper was
of considerable interest to me as I attempted a
similar, though far cruder in comparison, project
in my neural networks class last year. I was
ultimately unsuccessful, but this paper
demonstrates that the general idea is sound.

Personally, I believe the roles of these methods
are directly related to the complexity of the
task. For the simplest models, maybe a few joints,
an analytical solution can be very effective. The
challenge of constructing said solution quickly
becomes prohibitive as the complexity of the
linkage grows, and thus numerical solvers become
more popular. But I think as we have become more
expectant of good, ‘real-looking’ motion, a more
all-encompassing approach is needed. The database
driven system can look at the entire human body at
once, generating a pose that works over its
entirety. The major drawback of course is that you
need large databases for the figure type (ie. bipedal
verses quadruped ) and the its slowness makes it
unsuitable for true real time applications.

Danielle February 11, 2011 at 1:54 am

I read the Maciejewski, Footskate, and Style-Based IK papers.

It was interesting to see the evolution of IK computation and processing over this series of papers. It seemed as if the early paper sought to highlight a key problem in IK (structure can be problematic), and suggested solutions that applied new constraints to the simplified models discussed in several of the earlier papers we’ve read this semester. The second paper attempted to resolve some IK issues by using computational techniques and the third attempted to resolve IK issues by supplementing raw data with machine learning/statistical techniques. The “evolutionary” pattern is also interesting in that a number of the papers read for Readings 0 of this course involving locomotion appear to still by operating on this kinematics-by-example idea discussed in the third paper.

The Kovar paper brought up an interesting approach to solving a very concrete perceptual problem that could be done approximately real-time, making it appropriate for relatively dynamic scenarios (such as crowd simulation as cited in the paper). The technique’s usefulness is also demonstrated on a real scenario in motion capture data. The second technique, while asserting a certain desire to be real-time, feels as if it should be more computationally expensive, not necessarily in terms of the complexity of the calculations but rather the volume of computations needed to adequately summarize and derive inferences from the training data. Also, it is not developed for conformity to motion capture data, so I would imagine that this particular technique could not be used in strictly choreographed scenarios.

sandrist February 11, 2011 at 2:23 am

First I read the Maciejewski paper, which lays out the difficulty in IK created by the inherent ill conditioning of the problem. It was a good introduction to some of the math that can be involved. One thing I don’t have a good sense of is how relevant the solutions presented in this paper are today. Does the damped least-squares formulation still get used in practice?

The second paper I read was the footskate paper by Kovar. This paper gave an interesting method for solving the footskate problem by allowing for very slight changes in bone length. It seems to me like this would be extremely useful in applications of a more artistic nature, where just getting something that looks natural and “good” is the ideal. For a medical application where absolute fidelity is key, it perhaps might not be such a great idea to stretch bones.

The last paper I read was the Grochow paper. With no background in machine learning, this paper was a bit hard for me to parse. It seemed to me like they were getting good results performance-wise, but it was a little unclear to me how intuitive it would be to actually specify meaningful constraints and parameters in their system.

raja February 11, 2011 at 7:27 am

I read the Maciejewski, Footskate, and Wei papers.

Maciejewski’s paper was pretty tough for me to comprehend, though I get a much better “feel” of the numerical way of tackling the problem now.
Clearly, IK is not a linear problem, though we start off by using a gradient descent algorithm. Since the end effector’s end position & orientation (if it has both translational & rotational DOF) is a function of the DOF’s of all the joints in our skeleton, using a Jacobian matrix seems quite useful, wherein each entry refers to the effect on the end effector by changing a particular joint’s DOF.
The system could be over/under-constrained and pseudo-inverse Jacobian techniques are useful here; if the system were well-constrained, then the Jacobian inverse could be calculated. Both the above cases still suffer from degeneracy (when the derivative vectors line up). This is where SVD comes in. It allows us to decompose a matrix into a form that is easier to analyze and helps us know if it is singular/near-singular. The Jacobian transpose is also a cheap solution that compromises a bit on accuracy. The essence of numerical methods is in iteration and doing something until we deem it good enough.

The Footskate paper seemed to build on some of Mike’s previous work in Retargeting motions and used a really cool technique of manipulating bone length by moving the knee and ankles and the root a bit (w/o changing orientation) to prevent the sliding/slipping effect. Its also a separate post processing step and so it builds on other clean up mechanisms. It works on those frames with foot constraints (inferred automatically via previous work, though not the most accurate, thus requiring some manual editing). For the unconstrained frames, techniques like blending and displacement mapping are essential. I’m interested in knowing how these techniques don’t give rise to the footskate problem again (since they’re more of smoothing interpolation).

The Wei paper is the ” throw tons of data, infer stuff ” approach, allowing good deal of automation and coming up with pretty awesome results in real-time as well. The main disadvantage lies in the quality of data (importance of being non-redundant) though I feel this will be less of a concern if companies start sharing their mocap data. The main win here is that you don’t need to spend too much time in cleaning the data. I also think it can make use of previous runs to discard redundant data, making it more useful.

Analytical methods tackle simpler IK problems (or use assumptions that make it simpler) and less resource intensive and would be natural fit for games, though the data based approach also seems like something to build on. Numerical methods are cool to learn just because of the math and have a lot more power than the analytical methods. Data intensive methods which are parallelizable can be very useful both in speed and quality given a strong database and might be the favorite in the years to come.