This sessions goals:

  • Reconstruct Max.
  • Implementation of remote-control module.
  • Make remote controlled robot.
  • Make tightrope rider ride tight platform.
  • Calibrate PID control. Part2
  • Reconstruct Max with new motors
  • Calibrate PID control / Make Max balance.
  • Final conclusion.

The Plan

The plan for this session is mainly focused on making Max capable of balancing on a ball and also includes making the tightrope rider ride the tightrope platform that we built, as well as the making of the remote control module.

Project plan for session 6. Table 1: Project plan for session 6.

Reconstruct Max


As explained in session 5, we want to reconstruct Max into having a reversed pendulum construction. For this reason, we will rebuild him for this purpose.


The experiment will be quite simple. Because of the information we obtained in the previous session about center of mass in a reverse pendulum, we will place the NXT as high as possible, while at the same time provide a rigid construction.


The datalogger which we used in session 3 will be used for the accelerometer data collection.


The result can be seen in figure 1.

Max’ new body. Figure 1: Max’ new body.

Max has been built quite high (on the ball, he is more than 50cm in height) in order to use the theory of the inverted pendulum. We hope that this reconstruction will help us in making Max balance much better.

Implementation of remote-control module


Because we want to have an option for us to manually control where Max moves, we need a module in our code for that. This remote-control module will need to have a lower priority than the balance module, because we are not able to control anything if Max is not on the ball. We will use a computer as remote, which will let us send movement commands using the keyboard. We will need this module working before we can begin to implement the line follower. The line follower will be using the same commands as the remote will be sending.


We will implement the software necessary for the remote control to work and we will test this by using it to control our line following robot.


To communicate from the PC to Max we will use our bluetooth communication implemented in the last session. We have added extra code to handle the new functions.


Unfortunately, we have run out of time. The implementation of the remote control have been stopped to use our time on making Max balance so he is ready for the presentation.

Make tightrope rider ride tightrope platform


The plan for making the tightrope rider ride the tightrope platform consists of three different parts. First, we will deduce the most applicable colour for the line which it should follow, then calibrate the PID Control, and finally find the best and most reliable way of making a 180 degrees turn on the platform.


First we will deduce the most applicable colour for line tracking, which is done as seen in figure 2, by placing the four different colours that we have available on a white piece of thick paper, and then use the NXTCamView application to see which of the four colours are easiest to differentiate from the track and background.

Four available colours placed in front of the NXT Cam for comparison. Figure 2: Four available colours placed in front of the NXT Cam for comparison.

When comparing the tracking results from NXTCamView it was clear that both the black and the blue colours were too close to the background colours, e.g. the floor below the track, which resulted in causing irregularities. The yellow was too hard to discern from the white colour of the platform. The red colour was easily discernible and differentiated both from the track and background colours.

For calibrating the PID-Controller we used version 2 of the calibration GUI that we made in session 5 to calibrate the variables in real time. We, however, also had to calibrate the NXT camera using the NXTCamView application so that the colour of our red line was registered correctly which illuminated a problem with the track being in direct sunlight which made the red line merge with the white background.

The values that we found for our PID controller were as seen in figure 3.

Values for tightrope PID controller. Figure 3: Values for tightrope PID controller.

To get the tightrope rider to make a 180 degrees turn we looked in to two ways of doing this. First, we tried by making the track go around using the shape seen in figure 4.

Turn shape for tightrope rider. Figure 4: Turn shape for tightrope rider.

The shape did make the tightrope rider do the turn but since the rider tries to center itself on the line means that it is riding the edge of the track while turning and not turning on the spot, which caused it to fall over the edge a couple of times. This does, however, not require additional programming since it uses the same PID controlled steering as it normally uses.

The second way of making the turn is programmatically deduce a long red line perpendicular to the track line as seen in figure 5.

End of line-line. Figure 5: End of line-line.

By knowing when to turn 180 degrees programmatically means that we are able to make an actual rotation method which makes it possible to turn on the spot. The downside to doing this is that the way that the end-points are registered has to be consistent. During our testing we encountered episodes where the rider registered an end-line which was not there. This, however, showed us that it was able to do a 180 turn on the narrow part of the track.


Since the turn shape solution (Figure 4) for turning the rider, which was mentioned earlier, did not require additional programming all we will be describing in the this part is the programming needed to use the end of line-line (Figure 5).

To decide when the end of the track is found and when the rider is aligned with the track again is found using the following code snippet:

objectArea = camera.getRectangle(i);
if(objectArea.getWidth() > 76 && objectArea.getY() < 60){
  isEndReached = true;
  isOnLine = false;
}else if(95 < objectArea.getCenterX() && objectArea.getCenterX() < 105){
  isOnLine = true;
  isEndReached = false;
  isOnLine = false;
  isEndReached = false

What this means is that if the NXTCam finds an object with a width of more than 76 – which is the total width of the available space where we track within – and the placement of this object is less than 60 – which is the top of the camera image – then it is defined as the end of the track and the rider is told to turn around using the following code snippet:

  car.forward(baseSpeed, -baseSpeed, false);

The code snippet shown above is entered when the NXT camera detects an end it then stops the rider and starts turning right for at least 750ms. This is done to make certain that the camera does not register parts of the endline while turning and then aligns with the track again. The rider then waits to align with the track, stops and waits for 1000ms before resetting the values for the PID (which is the differential error, integral error etc. – not the constants controlled through the calibration GUI).

The complete code can be found as a part of the ball balancing bot source found here.


As seen in this video the rider was able to turn using the turn shape but was too inconsistent to be used as the final solution. We might have been able to alter the shape and calibrate the PID controller to such a degree that it would become a reliable solution but seeing as the rider is meant to represent Max balancing on a ball we decided that the correct solution would be to use the end lines.

By having end lines on a piece of hard paper we are able to dynamically change the endpoints on the track while the rider is running as seen in this video.

Calibrate PID Control – Part2


With the extra implementations in Max’s program and the alteration of his physical structure, Max is again ready to be tested and calibrated to balance. The plan in this exercise is basically the same as in the calibrate PID control exercise in session 5; to continuously calibrate the PID control to possibly get Max to balance. The difference that we plan for this exercise compared to the earlier exercises are adding an extra constant for calibrating the experiment; the alpha constant of the complementary filter. If Max is still not able to balance we are then able to conclude that the motors may not not powerful enough to correct Max’s balance.


The experiment for this exercise is similar to the one in the calibrate PID control exercise in session 5. Max will again be fitted with a safety rope for his new physical structure ensuring that he will not fall off the ball completely – saving us time from always repositioning him. The calibration will, again, be done in real time using the calibration GUI program which communicates with Max via bluetooth. The only difference with the actual calibration is that we additionally adjust the alpha constant from the complementary filter, which is described in session 5.


We managed to program the possibility of setting the alpha-value in the GUI. This is trivial so we will not comment it further.


Even with the new construction and complementary filter Max proved unable to balance with the current setup physically and programmatically. Max does not seem to be able to recover after tilting to any side. Even if the motors seem to respond to the tilt, they do not seem to be strong enough to get Max balanced again, since they do not seem to move but are instead locked when trying to move the opposite direction of Max’s tilt . Also after running for more than a minute, Max’s motor’s directions do not always correlate to Max’s tilted position, which results in one or three motors driving the opposite intended direction, which works against the goal of getting Max balanced.

With these results it seems that Max might need motors that have more torque and possibly the code to the tilt for each motors’ axes need to be corrected.

Reconstruct Max with new motors


Based on information from our previous experiments, we found that we need to replace the motors on Max as the current motors do not provide sufficient torque. We will change the motors in this exercise.


Looking back at session 2, we compared the various LEGO motors on different things. Additionally, we will now look at the comparison made by Philippe Hurbain, and in particular the different torque on the motors which can be seen in figure 6.

Comparison of torque between motors. Figure 6: Comparison of torque between motors.

As can be seen in Figure 6, the NXT is the motor with the most torque. Our current motors are the EV3-Medium, and as can be seen, the NXT has almost three times more torque (if you look at the website, the EV3-medium has 6,64 while the NXT has 16,7 For this reason, we want to use the NXT instead.


None in this session.


It was a difficult decision to take Max apart at first, but we believe it is the right decision to do so. As such, the new construction can be seen in figure 7.

Max N00b reborn! Figure 7: Max N00b reborn!

Now that we have the new NXT motors attached to Max, we are ready to give it a new try with an amazing PID-controller, and then finally have Max balance.

Calibrate Final PID Control


With Max rebuilt, we will now have our final calibration session to see if we can make Max balance on the ball.


With the new construction of Max our experiments will be looking into how we can calibrate the PID-controller for this. During our various trials, we found that the way we calculate the PID-control is not sufficient. We found this guide to change how the PID-calibration works, by looking into holonomic robots and how to calculate the speed for each individual wheel according to a specific direction in a two dimensional coordinate system.

Two-dimensional coordinate system for holonomic robots. Figure 8: Two-dimensional coordinate system for holonomic robots.

The theory, shown in figure 8 uses the movement of a joystick as an example where it measures the XY coordinate of the joystick tilt and calculates the length from the X-axis and Y-axis intersection to the XY coordinate, just like a vector. However, this gives a little problem as in the equation below shows.

With an equal given max length for X and max length for Y, the length of the vector will be the maximum of either X or Y, which does not fit together with the principle of a joystick, since the maximum length a joystick can move from the center is always the same in any direction.

The maximum distance of a joystick’s movement and the relation of the a vectors length with the maximum vector length in a XY coordinate system. Figure 9: Left: The maximum distance of a joystick’s movement. Right: The relation of the a vectors length and the maximum vector length in a XY coordinate system.

On the right of figure 9 the border for the maximum length will be drawn as a circle. So to translate the length from the vector to fit according to maximum length in the circle we will use the illustration on the right side of figure 9 and the the following equation, which is inspired from the slides and modified to work with angle tilt on the X and Y axis.

For further explanation, x and y will, on a joystick, be the coordinates of the stick itself and a is the vector length. For our robot Max, x and y will be the angle tilts for each of the two dimensions, so we basically translate the angles to coordinates. b will then be the maximum possible length for a vector in the two coordinates and the maxAngle is the maximum length of the angle within the circle. By using the above equations we can calculate the angles from the two dimensions into one angle at a specific direction. Thereby we will be able to know how much Max tilts in a specific direction. From here we just need to project the tilt angle to each specific motor as explained in the motor sensor relation exercise in session 4.


The changes in the new PID-controller can be found in the following code snippet:

public float projectedError(float forceY, float forceZ, double radians) {
  double force = Math.sqrt(Math.pow(forceZ, 2)+Math.pow(forceY, 2));
  double forceAngle = Math.atan2(forceZ, forceY);
  double angle = maxError*(force/maxForceYZ);
  float result = (float) (Math.cos(radians-forceAngle)*angle);
  return result;

As can be seen, the code takes tilt as forceY and forceZ and creates a vector and that vector’s force. However, since it is an angle of a direction according to a specific axis we are working with, and the maximum overall force cannot exceed the maximum force of either the axis forceY or forceZ (which are the same on both axes). We therefore divide the vector’s force with the maximum force of a vector for the maximum forceY and forceZ. The product of this, with the maximum error (the maximum angle we can get, which is a given constant), gives the exact angle of the tilt. This angle is then projected to fit with the specific motor according to the motor’s axis’ angle to the forceY axis.

The new MaintainBalance3 class, which contains the earlier explained implementation can be found here.


With the new changes to the projectedError() method and replacing the previous motors with NXT motors that have more torque, but lower rotations per minute, Max’ ability to balance has improved, as seen in this video.

The NXT motors’ extra torque help Max into recovering his balance better and making the wheels move against Max’s tilt. The change in the PID controller code also help significantly, since the motors consistently move in the direction desired.

Although these changes have proven a significant improvement, Max is still not able to balance on a ball by himself. Even through extensive tweaking in the PID constants, these are still not enough to keep him balanced. By observing his behavior with the changes and tweaks, there seem to be some factors that may inhibit Max’s balancing abilities.

One of these factors is Max’s reaction time from reading the sensor values and calculating the error, to giving a motor output and move on the ball. The reaction time may be too slow so Max cannot recover fast enough. There can be two obvious reasons to the slow reaction time, there can be calculations in the code that take a lot of time, or there can be limitations in the speed in which the sensors collect data.

Another factor is the calculation precision. Even though we measure Max’s angle in two dimensions and thereby know its tilted position according to the ball, we do not take Max’s angular velocity into account. The angular velocity can tell how fast Max is tilting in two dimensions, which in combination with Max’s given angle might also give a more exact calculation to how much Max shall move and thereby maintain balance.

Final Conclusion

We attempted to create a ball-balancing robot and the end result can be seen in this video. This project has been a collection of many small steps in order to achieve a higher goal. We have made a lot of small experiment to support each of our design choices when trying to assemble and program a ball balancing robot. We have looked into how the world of physics affects any and all ball-balancing possibilities and tried using the theory of inverted pendulums to help our robot, Max N00b, achieving ball-balancing abilities. To do this, we constructed a LEGO robot with three attached sensors; one accelerometers and two gyro sensor. We programmed a PID-controller for attaining balancing abilities which could be live-calibrated via a Java GUI that communicated with the NXT through Bluetooth. We wanted to make a ball-balancing robot that could also drive on a tightrope, and for this to happen we would need to create a behavioural software architecture, as discussed in session 3. Unfortunately, we never managed to make Max balance on the ball, and as such we did not finalize a complete solution that switched between behaviours as the balancing was not complete and the was the behaviour with the most priority. We did, however, program our software as if there was behaviour changes included, which means an arbitrator class is used to control the balancing behavior. We will now briefly discuss some of the issues we faced during our project, and future work to be done if we had more time.


LEGO Sensors

The sensors provided for the NXT by lego and third party are in many cases not designed to be used with high precision applications such as balancing on a ball. This makes sensing the movement of Max very difficult on a hardware level. A solution could be to use more precise sensors and then creating our own drivers to connect them to the LEGO NXT.

LEGO flexibility Because LEGO is plastic rods it makes it difficult to create a sufficiently rigid construction to handle the movement and forces which is affecting Max while trying to balance. A solution to this could be to glue the parts together which will remove some of the backlash present in the construction.

LEGO NXT limitations

We are not sure how much processing power the NXT has to offer when we start using more complex functions to calculate the balance point. It can be a problem when we have to use the camera tracking while the balancing function is running.

Future Work

Heavier ball

One of the things we would like to investigate further is how a heavier ball can affect the balance. We know from a master’s thesis that a heavy ball can make the ball drift less when trying to regain balance. We changed the motors in a later session to stronger motors which means we now have enough torque in our motors to use heavier balls to see how it may affect the balancing.

Other motors

We could also look further into the various motors available for LEGO. The current NXT that we use has the most torque of all motors, however it lacks speed. This can be an issue when the motor has to make quick changes in direction, so we would also like to test some of the different LEGO motors attached to Max and see how it would work on him.

More physics!

There are a lot of factors we do not account for in our implementation, for instance the rotation of the ball and how to counteract this. We can also include the resistance in the wheels to reduce the margin of error. We will also need to investigate if the size and the weight of the robot and the ball impact each other and how we can calculate these forces.

Sturdier structure

Since the project was about creating a ball-balancing bot made out of LEGO it means that we are limited to what we can do. Gluing the parts together is of course not possible since Max has to be disassembled and used for other projects once this course has ended. Therefore trying to rethink the construction made and maybe supplying it with a couple of zip ties could be the solution for a more sturdy construction.