JaeOS® -- Just Another Embedded OS
Because the RTOS should be a commodity.
JaeOS® User Guide
This document has been automatically converted from an MS Word document. Formatting may look different from normal web pages.
Contents
Symmetric Multiprocessing (SMP)
Restricting Tasks to Certain CPUs
Introduction
The main purpose of JaeOS is to be a commodity embedded OS. That is to implement some common system calls found in most Real Time Operating Systems (RTOS) that allows software packages that require some "generic" RTOS support to run on it.
The name JaeOS stands for “Just Another Embedded Operating System” acknowledging the fact that there are many similar operating systems out there. Some of them look like unfinished school projects where the author wants to show off all the different things that can be part of an RTOS but never quite get to finish everything. Others are well supported commercial products with the kitchen sink included, that come with very restrictive licenses and a hefty price tag. There is nothing wrong with using them, especially if the end product requires the functionality what they provide.
But a lot of projects really do not need all that functionality and some organizations don"t like to pay a lot of money for something that is at most a few thousand lines of code, especially if they only need a small portion of its functionality.
People do all sorts of seemingly irrational things to avoid having an RTOS in the product, including inventing all sorts of ad hoc solutions to try to cram functionality into a single threaded system even when the problem clearly needs parallelism.
JaeOS is also intended to make RTOS accessible to people who usually want to run things on bare metal systems, but find themselves working on projects that really require some amount of multitasking and would benefit from having an OS.
Note: JaeOS is copyrighted material distributed under an MIT style license that permits commercial use without restriction. It does not however come with any warranty. It is strictly the responsibility of the user to verify that it fits the needs of a particular project or if its performance is appropriate for that project.
Core Functionality
I have tried to keep the JaeOS core as simple as possible. Although there are optional features that are somewhat more complex the base RTOS system is close to bare minimum. For example instead of having the half a dozen or so synchronization primitives found in various RTOS systems, JaeOS only has counting semaphores – which are quite usable as binary semaphores or mutexes, locks, etc.
Most API functions only have one version to be called from task or ISR context – perhaps with some restriction on what combination of parameters are permitted in each case.
It was a design goal that at least the core functionality should be fairly trivial to understand.
Some limitations on e.g. maximum number of supported tasks were deemed acceptable so long as the OS can support realistic common embedded applications.
Because the target system are small embedded applications JaeOS neither provides nor requires dynamic memory allocation functions (the equivalent of malloc() and free()). JaeOS can be used with dynamically allocated memory, but it does fine without it.
Scheduling
A little bit of implementation details to help better understand some limitations of JaeOS.
In JaeOS tasks have a unique index which also serves as their priority (thus each task has its own unique priority – there is no way to declare multiple tasks at the same priority).
Sets of tasks representing such things as "all tasks ready to run" or "all tasks waiting for a semaphore" are represented as a bitmap, basically bits in some integer type. The actual type varies depending on the target processor as well as the number of tasks actually present in the system, but for most modern processors uint32_t is efficient (and narrower types might be more expensive to handle) while wider types may be possible if uint64_t or even uint128_t is supported by the compiler.
|
Bit Position |
0 |
1 |
2 |
3 |
4 |
… |
… |
31 |
|
Task |
Idle |
LowPrio |
HighPrio |
- |
- |
- |
- |
- |
|
Bit Value |
1 |
1 |
1 |
0 |
0 |
0 |
0 |
0 |
These types are easy to work with and they result in efficient code. Even operations such as finding the first bit that is set are supported as a single instruction on most modern processors. Thus if one can accept living with a limitation of how many tasks are supported by the OS using integers as bitmaps to represent sets of tasks is a simple and efficient way to implement the concept of "a set of tasks" and result in easy to read code.
A lot of embedded systems that I have been working on in the past had between half a dozen to a dozen tasks, thus having a built in limitation in the system of only being able to handle at most 32 (or 64, or 128) tasks did not seem serious at all. Thus in spite of the fact that computer science purists might find it undesirable, JaeOS uses tasks sets implemented bitmaps in an integer extensively, and it has the limitation of only being able to support a limited number of tasks as described above.
In practice that should be all one ever needs in the type of systems JaeOS is aimed at. If you really need a large number of tasks you probably do need a bigger OS. I just seem to end up working on systems that require around a dozen or so tasks altogether, so I have to assume that is a perfectly valid use case.
When assigning task priorities you should only define as many as you need and try to keep the highest priority at 31 or less.
Priority Based Scheduler
JaeOS uses a priority based real time scheduler. It means that the scheduler will always pick the highest priority task that is ready to run and let it run as long as it wants to or until a higher priority task becomes ready to run. This is the normal scheduler that is run after interrupt processing and when some API function performs an operation that might change which tasks are ready to run.
|
Bit Position |
0 |
1 |
2 |
3 |
… |
31 |
|
Task |
Idle |
LowPrio |
HighPrio |
- |
- |
- |
|
Bit Value |
1 |
1 |
1 |
0 |
0 |
0 |
|
Selection. |
|
|
Highest 1 is selected to run. |
Higher bit positions are all zeroes. |
||
Yielding Scheduler
The yielding scheduler is triggered only if the task explicitly calls RTOS_YieldPriority() in order to temporarily relinquish the CPU. Its use is described in more detail in the section “Temporarily Yielding the CPU”.
The Yielding Scheduler doesn"t always pick the highest priority task that is ready to run, but selects a task for running based on the following algorithm:
Select the highest priority task that is ready to run and has a lower priority than the current task but only if it is not the idle task (we do not want to give up the CPU just to put it into idle).
If no task satisfies the requirement above just act as the normal priority based scheduler and select the highest priority task that is ready to run (which can be the current task or a higher priority task if the current task was selected by a previous yield from another task). This way a number of tasks yielding to each other will cycle round and round, possibly switching tasks a lot of times but each having a chance to make progress.
Locking the Scheduler
From its inception JaeOS has supported locking the scheduler – which means preventing the scheduler from selecting a new task to run. This feature can be used to briefly run sections of code as single threaded.
The interface consist of the functions RTOS_LockScheduler() and RTOS_UnlockSchduler(), these calls are nestable.
To enable this feature the following line has to be present in rtos_config.h:
#define RTOS_INCLUDE_SCHEDULER_LOCK
While I can think of some rare situations when having the ability to lock the scheduler is useful, it seems to be almost always a bad idea to do so. I have decided not to remove scheduler locking at this time, but one should avoid using it if there is some more proper solution for a problem.
Because disabling task switching is nearly utterly useless during in multiprocessing mode when other tasks still can be running on other CPUs even if the scheduler is disabled, scheduler locking is not supported in SMP mode.
Critical Sections
Like most Real Time Operating Systems, JaeOS provides critical sections.
Being in a critical section means exclusivity in the sense that no two pieces of code can be executing inside a critical section at the same time. Not another task (not even one running on a different CPU in SMP mode), and not an interrupt service routine (ISR).
Critical sections are implemented by disabling interrupts, thus in single processor systems being in a critical section means that really nothing else can run. All existing targets disable interrupts during interrupt processing (do not allow interrupts interrupting interrupts) so interrupt service routines are implicitly run as critical sections. This gives a simple and robust design.
If necessary this could be eliminated by crafting the target specific code with some finer grain control over which parts are critical, there is nothing inherent in the core JaeOS design to prevent that, but ISR design may become somewhat more complex, or on some processor architectures nearly impossible, if interrupts are to be allowed to interrupt other interrupts.
I personally prefer simplicity and just make ISR execution exclusive but keep interrupt service routines short and do any lengthy work in task level code.
A special case is Symmetric Multiprocessing mode (SMP) because even if interrupts are disabled on a processor core something can be executing on another core.
While it is dependent on the implementation for specific targets, the preferred way to implement critical section nesting in JaeOS is a save and restore model. I.e. Upon entering a critical section the machine status register (or whatever it may be called on the target system) is saved and then interrupts are disabled, and exiting a critical section just restores it to its previous state thus making critical sections nestable. There are alternative strategies, such as counting how deeply critical sections are nested in a global variable, and it is possible to design a target port for JaeOS which does that.
Note: In an RTOS task scheduling is still possible inside a critical section if the scheduler is explicitly invoked by the executing code. Critical sections are per task, thus a task that is inside a critical section can cause a task switch to another task that is not in a critical section, thereby suspending its critical section. When the first task is scheduled to run again the critical section is restored.
The interface to use critical sections is quite simple:
void some_function(void)
{
RTOS_SavedCriticalState(saved_state); // Declare variable to save previous state in.
RTOS_EnterCriticalSection(saved_state); // Enter critical section saving previous state.
SomethingTahAbsolutelyNeedsToBeUninterrupted();
RTOS_ExitCriticalSection(saved_state); // Restore previous critical state.
}
Because they stop most things in the system critical sections should only be used for a good reason. If a task needs to alter the system"s state in some way that could end up in an inconsistent if interrupted it might have to do that inside a critical section. Or if one is talking to hardware and a number of hardware registers need to be written and e.g. it is time sensitive (cannot have arbitrary delays between writes) doing it in a critical section might be a good option.
There are other uses such as updating some variable(s) shared by other tasks really quickly.
But for any lengthy operation on data shared between tasks a proper synchronization mechanism, such as a semaphore used as a lock should be utilized. Do not create lengthy critical sections because the degrade the performance of the entire system.
Tasks
In a multi-tasking operating system code is executed either as part of an Interrupt Service Routine (ISR) of in the context of a task. A task is what "grown up" operating systems call a thread of execution. Unlike full featured OS-es such as Linux, a lot of real-time operating systems don"t require an MMU. Tasks run in the same memory space, usually compiled into one executable binary image, which means they are not physically protected from each-other.
For small applications that are developed by one organization and don"t allow user installed software this is perfectly adequate because there aren"t uncooperating applications trying to wreak havoc on the system. The typical small embedded application built on an RTOS is just like a single threaded application but with the added functionality that various parts of the system can run at the same time. It is not at all like running a Un*x type system such as Linux (or its derivatives such as Android) or Windows.
Typical RTOS based systems are close to bare metal, but handle concurrency in an organized manner instead of using some ad hoc solution invented for one application.
In a multi-tasking operating system such as an RTOS tasks appear to be running at the same time. This is usually – with the exception of some SMP systems – is an illusion. The same CPU is executing all the tasks quickly switching between them so they all appear to progress in parallel.
Each task has a separate stack area but the address space is typically shared among tasks as they are just different sections of the same executable. Thus certain things such as global variables or hardware peripherals are visible from all tasks. Also there is no distinction between user space and kernel space as it would be in Linux or Windows.
In theory tasks could just access any data they want, but in reality reading and writing global variables in an arbitrary fashion will result in unpredicable behavior. E.g.: If two tasks were trying to increment a global variable without synchronization at roughly the same time the variable might end up incremented only one or twice depending on when tasks switching happen between read and write operations. Such seemingly random behavior is undesirable, so operating systems provide ways for tasks to communicate with each other and synchronize their actions.
The following sections describe the beasics of how to create tasks and work with them in the JaeOS RTOS.
Creating Tasks
In JaeOS a task is executing a C function, the tasks has a unique priority level (i.e. each task has its own priority) and a separate stack. Each system running JaeOS must have an idle task which has a fixed priority of 0. This idle task is executed by the OS if none of the other tasks are ready to run because they are sleeping or waiting for some event or resource before they can progress.
Thus probably the simplest system will have at least two tasks, the idle task and something that does some work.
So to have a "Hello World!" example first we need to define the priority levels that we are going to use.
As a bare minimum we need something like the following in rtos_config.h:
#define RTOS_Priority_Hello 1
#define RTOS_Priority_Highest RTOS_Priority_Hello /* The highest priority task. */
#define RTOS_TASK_NAME_LENGTH 32
Then create a C source file (main.c) with the contents below. In this example Board_Puts() stands for a function that can print a string on some peripheral that is visible to the user, which can be a serial port or some sort of display. A suitable prototype or macro would be defined in board.h, for the sake of this example we do not care at the moment how it is implemented.
Tasks are created by the function RTOS_CreateTask(). Its parameters are the address of the task structure to be initialized, the priority of the task, the address of the stack space for the task, the stack size (how many items will fit on the stack), the main function of the task and a parameter of type void * to be passed to the newly created task. It normally returns RTOS_OK for success, RTOS_ERROR_OPERATION_NOT_PERMITTED if the task is NULL, RTOS_ERROR_PRIORITY_IN_USE if another task has already been assigned to the same priority level, and RTOS_ERROR_FAILED if there is some other problem, e.g. the stack location or size is inappropriate.
#include <rtos.h>
#include <board.h>
#define STACK_SIZE 512
RTOS_StackItem_t stack_hello[STACK_SIZE];
RTOS_StackItem_t stack_idle[RTOS_MIN_STACK_SIZE];
RTOS_Task task_hello;
RTOS_Task task_idle;
// Say Hello!
void say_hello(void *p)
{
Board_Puts("Hello World!\r\n");
(void)p; // Pacifier for the compiler.
}
int main()
{
RTOS_CreateTask(&task_idle, RTOS_Priority_Idle,
stack_idle, RTOS_MIN_STACK_SIZE, &RTOS_DefaultIdleFunction, 0);
RTOS_SetTaskName(&task_idle, "Idle");
RTOS_CreateTask(&task_hello, RTOS_Priority_Hello, stack_hello, STACK_SIZE,&say_hello, 0);
RTOS_SetTaskName(&task_hello, "Hello");
Board_HardwareInit(); // Initialize hardware as appropriate for the system/board.
RTOS_StartMultitasking(); // Start JaeOS.
Board_Puts("Something has gone seriously wrong!\r\n"); // We should never get here!
while(1);
return 0; // Unreachable.
}
This will print the string “Hello World” once and then the "Hello" task returns and only the idle task is run. This example demonstrates a property of JaeOS which is not the same among all real time operating systems: In JaeOS if a task has finished its work and it does not need to run again it can just return from its main function. No explicit kill operation is necessary.
A more usual way to do things in an embedded system is that tasks remain running forever (well, while the device it is running on is powered) thus the typical main function of each task is an endless look.
Our hello world example would look like this with an endless loop, printing the string “Hello World!” forever:
// Say Hello!
void say_hello(void *p)
{
while(1)
{
Board_Puts("Hello World!\r\n");
}
(void)p; // Pacifier for the compiler.
}
Sleeping and Delaying
Typically higher priority tasks are expected to do their work and then wait for new work to arrive letting other tasks run. Waiting is often accomplished by synchronization constructs such as waiting for a semaphore to be posted.
Other times a task might just want to do things periodically, sleeping in between for some short (or sometimes longer) periods of time.
The simplest way to accomplish that is to sleep for a certain number of timer ticks where a timer tick is the time between two timer interrupts. The function that does that is RTOS_Delay() which has the following prototype:
RTOS_RegInt RTOS_Delay(RTOS_Time ticksToSleep);
The task that calls RTOS_Delay(ticks) is put to sleep and it will not be scheduled until ticks number of timer interrupts take place. If a 1 tick delay is requested the task will sleep some unspecified small amount of time smaller than the time between two timer interrupts – that is because it goes to sleep somewhere between two timer interrupts and the first timer interrupt will wake it up.
If 0 ticks are specified RTOS_Delay() returns immediately.
The return value is usually RTOS_OK indicating that the sleep has taken place. If the task is woken up prematurely by RTOS_WakeupTask() then RTOS_ABORTED is returned. Not all applications use RTOS_WakeupTask() so checking the return value of a delay operation might or might not be necessary. In general RTOS_Delay() can be assumed to have successfully slept the requested amount of time unless the application specifically performs actions that can wake up sleeping tasks prematurely. Thus most application will not check the return value of RTOS_Delay() and it is usually safe to do so.
In JaeOS there is another delay function:
RTOS_RegInt RTOS_DelayUntil(RTOS_Time wakeUpTime);
It is very similar to RTOS_Delay() but instead of the number of ticks to sleep the actual time when the task needs to be waken up is specified.
Its use is if a task needs to perform some operation at some equal intervals.
If one wanted to implement a function that performs some work every 100 ticks the first approach would be to just do this:
void f(void *p)
{
while(1)
{
RTOS_Delay(100);
DoSomeWork();
}
}
The problem with it is that this does not perform the work at 100 tick intervals but some unspecified and possibly variable time longer interval, depending on how long DoSomeWork() takes.
If one wants a more accurate timing RTOS_DelayUntil() can help with that:
void f(void *p)
{
RTOS_Time timeToWakeUp = RTOS_GetTime() + 100;
while(1)
{
RTOS_DelayUntil(timeToWakeUp);
DoSomeWork();
timeToWakeUp += 100;
}
}
This version always wakes the task up 100 timer interrupts after the last time it was woken up, thus spacing the start of DoSomeWork() more evenly, independent of how long each run of DoSomeWork() takes.
Note: The example above assumes that the system is not under so heavy load that DoSomeWork() will not even finish within 100 ticks. Also the task must have high enough priority so that it is not blocked by other tasks or the timing cannot be guaranteed.
Care must be exercised when using RTOS_DelayUntil(wakeUpTime) because it treats its parameter as a time in the future (does not have the concept that the specified wakeUpTime is in the past). This is to allow proper operation even if the system time is wrapping around.
E.g.: If the system time is 0xFFFFFFFF and wakeUpTime is specified as 1 it does not mean to wake up immediately (as one would think because 1 < 0xFFFFFFFF) but to wait until the system time becomes 1, which, if the system time is a 32bit unsigned integer, means waiting for two timer interrupts to come in.
RTOS_Delay() and RTOS_DelayUntil() cannot be called from inside an Interrupt Service Routine (ISR) or if the scheduler is locked. If the application attempts to do that RTOS_ERROR_OPERATION_NOT_PERMITTED is returned and the sleep does not take place. Again, depending on the structure of the application this may or may not be a concern, so checking the return value may not be strictly necessary.
In order to have the delay functions available rtos_config.h must have the following line:
#define RTOS_INCLUDE_DELAY
Temporarily Yielding the CPU
While tasks are discouraged to perform any sort of busy waiting, there can be legitimate reasons to poll for some condition. Also programmers, perhaps out of sheer laziness, do use busy waiting perhaps more often than they should. Since in a multi-tasking environment one does not want to burn CPU cycles while polling for something, a way to relinquish the CPU so that other tasks can run is needed. A simple way to do that in an RTOS is to perform a short delay like so RTOS_Delay(1); -- this will release the CPU in effect until the next timer tick interrupt.
While usable this is inefficient since it puts the task to sleep even if there is no other useful task to perform, yet sometimes unavoidable – unless one writes proper code and uses synchronization to wake up the task at the right time, but as I said people don"t always do that.
In order to accommodate reality JaeOS provides a system call to "relinquish the CPU if someone else has better things to do". It is called RTOS_YieldPriority(), no parameters or return value. It triggers the execution of the yield scheduler that can pick a task to run that is not the highest priority task as described in the Scheduler section of this document.
RTOS_YieldPriority() is always available if the target port supports it.
Note: Scheduling evens such as interrupts, or posting a semaphore, or calling a delay function will trigger the call of the normal scheduler. Thus if a task is yielding the CPU to some other lower priority task and the other is performing work in a tight loop the first task will get the CPU back when either an interrupt or another scheduling event takes place.
Prematurely Waking up Tasks
To forcefully wake up a task that is sleeping or waiting for a semaphore/event the function RTOS_WakeupTask() is provided:
RTOS_RegInt RTOS_WakeupTask(RTOS_Task *task);
This function is only included if the following line is present in rtos_config.h:
#define RTOS_INCLUDE_WAKEUP
Calling RTOS_WakeUpTask(task) will prematurely awaken a sleeping or waiting task.
Return values: RTOS_OK indicates success. If the task cannot be awakened because the task parameter is NULL or if the task is not actually sleeping RTOS_ERROR_OPERATION_NOT_PERMITTED is returned.
RTOS_WakeUpTask() is a scheduling point, i.e. it triggers the scheduler and if the task that was just forcefully woken up has a higher priority than the caller of RTOS_WakeUpTask() it is scheduled immediately.
The usefulness of RTOS_WakeUpTask() is somewhat limited, it just exposes existing functionality, namely the ability to forcefully wake up a sleeping task, that is present internally in JaeOS.
Suspending and Resuming Tasks
To have suspend and resume functionality the following line needs to be included in rtos_config.h:
#define RTOS_INCLUDE_SUSPEND_AND_RESUME
Suspending a task makes it dead as far as the operating system is concerned. A suspended task is no longer running or sleeping, or waiting for some event, it is just doing nothing. The only reason why it is not entirely dead because it can be resumed and continue operation.
To suspend a task use:
RTOS_RegInt RTOS_SuspendTask(RTOS_Task *task);
The specified task will be suspended. If it was running or ready to run it just gets suspended. A task can suspend itself, and in fact that seems to be almost the only sensible use of suspend. If a task that is sleeping or waiting is suspended it is forcefully woken up first as if by RTOS_WakeUpTask() and then placed into suspended animation.
Suspended tasks can be restarted by RTOS_RegInt RTOS_ResumeTask(RTOS_Task *task); a resumed task is ready to run (can be picked by the scheduler to run). If the task was waiting or sleeping when it was suspended it returns from the function that caused it to sleep or wait with the return value RTOS_ABORTED.
Return values: RTOS_OK on success. RTOS_ERROR_OPERATION_NOT_PERMITTED if task is null.
JaeOS does implement suspending arbitrary tasks while they are ready to run or are sleeping, waiting for an event/semaphore etc. in case some software package that requires an RTOS underneath it requires such functionality. But in reality most obvious uses only require the functionality that a task should be able to suspend itself.
Both suspend and resume operations trigger the call of the scheduler if called from task context (not from an ISR).
Deleting Tasks
In JaeOS the easiest way to stop a task is to return from its top level function. The operating system will consider the task to be dead and its priority level reusable. There is no real resource management in JaeOS such as allocating ownership of resources to a particular task so there really isn"t a notion of freeing those resources if a task is no longer executing.
In order to satisfy software libraries that expect an explicit way to kill a task the function RTOS_KillTask(task) is provided.
RTOS_RegInt RTOS_KillTask(RTOS_Task *task);
To include it in the OS define RTOS_INCLUDE_KILLTASK in rtos_config.h.
In theory a task can be killed no matter what it is doing, but it is always a messier affair than a task exiting on its own terms. It is highly recommended to not design software that requires killing arbitrary tasks. I can"t really see any legitimate use for RTOS_KillTask(), it is provided strictly because some OS compatibility layers require it. In all fairness many applications only use it to allow a task to kill/delete itself, which is a valid use case.
Time
Time in JaeOS is measured in timer ticks. The system time starts at 0 and it is incremented at each timer interrupt.
In order to do that the function rtos_TimerTick() needs to be called from the ISR that services the timer interrupt on the CPU.
The actual details of how to set up an interrupt handler is dependent on the hardware platform JaeOS is ported to and has to be dealt with in the target specific code. It is a low-level hardware specific thing, not a generic “RTOS thing”.
The system time is kept in a variable of type RTOS_Time_t which default to a 32-bit unsigned int. If an application of a specific target system requires a different type, the macro RTOS_TIME_TYPE can be used to override it.
E.g.: Put a line like the following into rtos_config.h:
#define RTOS_TIME_TYPE uint64_t
The system time can be read by calling the following function:
RTOS_Time RTOS_GetTime(void);
The system time starts at zero when the OS is started and it increments by 1 at each call to rtos_TimerTick().
The macro RTOS_TICKS_PER_SECOND defines the number of timer ticks per second. It can be defined in rtos_config.h, if omitted it defaults to 100. Hardware setup routines should make sure that the timer tick actually happens this many times a second.
In order to be able to define time in terms of milliseconds instead of ticks the following macros are defined to do the conversion:
RTOS_MS_TO_TICKS(MS)
RTOS_TICKS_TO_MS(TICKS)
Dealing with checking all sleeping tasks if their timer has expired and possibly waking them up takes time. Even though on most modern processors it is not a lot of time some might still consider that it is too much code to run inside the timer ISR preventing other interrupts from coming in.
JaeOS offers the alternative to run most of the work in a task, allowing interrupts to be serviced. Also while the timer task has to have a reasonably high priority it does not need to have the highest priority in the system, thus some really urgent task level work can be performed by a higher priority task so long as it is done reasonably quickly.
To move most of the work from the timer ISR to a task define the following in rtos_config.h:
#define RTOS_USE_TIMER_TASK
#define RTOS_Priority_Timer 7 /* The priority of the timer task (7 is just an example) */
The application code also has to create the actual timer task:
int main()
{
….
#if defined(RTOS_USE_TIMER_TASK)
RTOS_CreateTask(&(tasks[RTOS_Priority_Timer]), RTOS_Priority_Timer,
&(stacks[RTOS_Priority_Timer]), STACK_SIZE, &RTOS_DefaultTimerFunction, 0);
RTOS_SetTaskName(&(tasks[RTOS_Priority_Timer]), "Timer");
#endif
…
}
Synchronization
Semaphores
JaeOS offers a limited number synchronization primitives. Basically only counting semaphores are provided because they can be used instead of most other constructs such as mutexes/locks but not the other way around. The underlying event handling is also exposed ("naked" events – i.e. not wrapped into any of the usual synchronization primitives).
To enable semaphores define RTOS_INCLUDE_SEMAPHORES in rtos_config.h.
Here is a full example:
#ifndef RTOS_CONFIG_H
#define RTOS_CONFIG_H
#define RTOS_TASK_NAME_LENGTH 32
#define RTOS_TICKS_PER_SECOND 100
#define RTOS_INCLUDE_SEMAPHORES
#define RTOS_INCLUDE_DELAY
#define RTOS_Priority_Task1 1
#define RTOS_Priority_Task2 2
#define RTOS_Priority_Highest RTOS_Priority_Task2
#endif
And the corresponding main.c:
#include <rtos.h>
#include <board.h>
#define STACK_SIZE 512
RTOS_StackItem_t stack[2][STACK_SIZE];
RTOS_StackItem_t stack_idle[RTOS_MIN_STACK_SIZE];
RTOS_Task task[2];
RTOS_Task task_idle;
RTOS_Semaphore s;
void high_prio(void *p)
{
RTOS_RegInt res;
while(1)
{
res = RTOS_GetSemaphore(&s, RTOS_TIMEOUT_FOREVER);
if (RTOS_OK == res)
{
Board_Putc('+');
}
else
{
Board_Putc('-');
}
}
(void)p; // Pacifier for the compiler.
}
void low_prio(void *p)
{
while(1)
{
RTOS_PostSemaphore(&s);
Board_Putc('>');
RTOS_Delay(RTOS_MS_TO_TICKS(1000));
}
(void)p; // Pacifier for the compiler.
}
int main()
{
RTOS_CreateTask(&task_idle, RTOS_Priority_Idle,
stack_idle, RTOS_MIN_STACK_SIZE, &RTOS_DefaultIdleFunction, 0);
RTOS_SetTaskName(&task_idle, "Idle");
RTOS_CreateTask(&(task[0]), RTOS_Priority_Task1, stack[0], STACK_SIZE, &low_prio, 0);
RTOS_SetTaskName(&(task[0]), "Task1");
RTOS_CreateTask(&(task[1]), RTOS_Priority_Task2, stack[1], STACK_SIZE, &high_prio, 0);
RTOS_SetTaskName(&(task[1]), "Task2");
RTOS_CreateSemaphore(&s, 0);
Board_HardwareInit(); // Initialize hardware as appropriate for the system/board.
RTOS_StartMultitasking();
Board_Puts("Something has gone seriously wrong!\r\n"); // We should never get here!
while(1);
return 0; // Unreachable.
}
If run this program produces output similar to the following:
+>+>+>+>+>+>+>+>
Notice that the first character is printed by the high priority task which runs as soon as the low priority one posts the semaphore. The low priority task continues to run after the high priority one blocks again waiting on the semaphore, and so on.
The functions used to accomplish that are the following:
RTOS_RegInt RTOS_CreateSemaphore(
RTOS_Semaphore *semaphore,
RTOS_SemaphoreCount initialCount
);
RTOS_CreateSemaphore() initializes a semaphore (turns "raw memory" into a semaphore. Strictly speaking this example would work without it because a static variable in C is set to all 0-s and that is a valid semaphore with a count = 0. But there are cases when one wants to create a semaphore with an initial count other than 0. E.g. If the semaphore is to be used as a lock it is usual to create it with a count = 1 (unlocked). Also if the semaphore is located in something other than the static variable area, e.g. inside some structure that was malloc()-ed it needs to be properly initialized before it can be used.
RTOS_RegInt RTOS_GetSemaphore(
RTOS_Semaphore *semaphore,
RTOS_Time timeout
);
RTOS_GetSemaphore() checks if the semaphore has been posted (its counter is not 0), if the counter is non-zero it is decremented by one and RTOS_GetSemaphore() returns success (RTOS_OK). If however the semaphore"s counter is zero the actions performed as follows: If the timeout value is 0 it means we are unwilling to wait and RTOS_GetSemaphore() immediately returns RTOS_TIMED_OUT. Otherwise the calling task is added to the set of tasks waiting for the semaphore and it is removed from the set of ready to run tasks. If timeout is specified as RTOS_TIMEOUT_FOREVER this is the only action, otherwise the task is also set to be woken up the same way as if it has called RTOS_Delay() with the same number of timer ticks as the timeout value passed to RTOS_GetSemaphore().
If the semaphore is posted before the timeout expires the caller of RTOS_GetSemaphore() is woken up and RTOS_GetSemaphore() returns RTOS_OK. If multiple tasks are waiting for the same semaphore only the highest priority task that is waiting is woken up. Priority based scheduling means that higher priority tasks are always more important than lower priority ones so they get serviced first regardless in what order they have started waiting.
If the timeout expires before the task is woken up by posting the semaphore RTOS_GetSemaphore() returns RTOS_TIMED_OUT.
I.e.: The normal return values are RTOS_OK for success and RTOS_TIMED_OUT if the semaphore cannot be decremented within the specified timeout limit.
JaeOS allows RTOS_GetSemaphore() to be called from an ISR or while the scheduler is locked, but in either of these cases it cannot actually wait. If it would have to wait RTOS_ERROR_OPERATION_NOT_PERMITTED is returned. In order to have a consistent behavior if an ISR has to call RTOS_GetSemaphore() it should always specify timeout = 0, in which case if the semaphore"s count is zero, RTOS_TIMED_OUT is returned. I.e: The return value is consistent with other use cases: RTOS_OK for success and RTOS_TIMED_OUT if a non-zero count is not available within the specified time limit (in this case 0, i.e. immediately).
RTOS_RegInt RTOS_PostSemaphore(RTOS_Semaphore *semaphore);
This is the function that posts a semaphore. If there are tasks waiting for the semaphore the highest priority one is woken up. If nobody is waiting for the semaphore the semaphore"s counter is incremented so a subsequent call RTOS_GetSemaphore() to will know that the semaphore has already been posted and just decrement the counter instead of waiting for a post.
The return value is RTOS_OK on success, RTOS_ERROR_OPERATION_NOT_PERMITTED if the semaphore parameter is NULL, and RTOS_ERROR_OVERFLOW if the semaphore"s counter has already reached its maximum value and it cannot be incremented.
RTOS_PostSemaphore() is safe to call from an ISR or when the scheduler is locked because it does not cause the caller to wait. In fact a rather common use case is that a task is waiting for a semaphore which is posted from an ISR when a peripheral needs attention. I.e.: The only action performed inside the ISR apart from acknowledging the interrupt is to post a semaphore. The actual work is performed by a task. Among other things this allows most of the work to be done with interrupts enabled in case something more urgent is happening in the system.
The semaphore API has one other noteworthy function:
RTOS_SemaphoreCount RTOS_PeekSemaphore(RTOS_Semaphore *semaphore);
It returns the value of the semaphore"s counter.
The justification for the existence of this somewhat unusual function is that counting semaphores can be used to count events, or items placed into some container, etc. I do strongly believe that one piece of information should be maintained only in one place. So if it turns out that a semaphore is the appropriate device to count something (because of its synchronization properties) it is logical that one would want to know the actual value of its count.
Low-level Events
Semaphores could have been implemented in a somewhat monolithic way, but the mechanism used for making a task wait is factored into generic "event" functions that could be used to implement other synchronization primitives.
Over the years when writing code for other RTOS-es I have often wished that there was a way to wake up a task when something happens in the future (but not in the past), or if nobody is interested simply ignore the event altogether. One can perhaps solve the problem at hand through the creative use of suspend and resume but I always would have liked to have a more organized way to do it.
What I really wanted is a semaphore without a counter.
But the underlying event handling in JaeOS that semaphores are built upon does exactly that.
So I have exposed it as part of the API as "naked" events – i.e. not wrapped into any of the usual synchronization primitives.
To include this API #define RTOS_INCLUDE_NAKED_EVENTS in rtos_config.h.
The following three functions are available to the application programmer:
RTOS_RegInt RTOS_CreateEventHandle(RTOS_EventHandle *event);
RTOS_RegInt RTOS_WaitForEvent(
RTOS_EventHandle *event,
RTOS_Time timeout
);
RTOS_RegInt RTOS_SignalEvent(RTOS_EventHandle *event);
These are similar to the semaphore functions with the notable difference that there is no counter. Signaling an event wakes up the highest priority task waiting for the event, if there is nobody waiting it does nothing. The return value of RTOS_SignalEvent() is RTOS_OK if there was a task waiting and RTOS_TIMED_OUT if there were no tasks waiting for the event at this time.
RTOS_WaitForEvent() returns RTOS_OK if the event is signalled and RTOS_TIMED_OUT if the timeout has expired without the event being signalled. A timeout value of zero returns immediately with RTOS_TIMED_OUT, it performs no useful operation – but this interface is consistent with other wait functions. A timeout value of RTOS_TIMEOUT_FOREVER means to wait until an event comes in and do not time out.
Miscellaneous Settings
There are some other configurable options in JaeOS to give the user a finer control over error checking.
Disabling Some Sanity Checks
#define RTOS_DISABLE_RUNTIME_CHECKS
It disables a number of runtime checks such as verifying if parameters passed to a function are not NULL and if something was called in the right context (e.g. something that would wait in not inside an ISR and the scheduler is not locked).
In general I do not like the idea of not checking a function"s parameters, but the checks do require extra instructions to be run and if one is cutting time really close every little bit counts. If for example the code can be verified, e.g. by static code reviews that it does not e.g. try to call some wait or delay function inside an ISR the check can very well be considered unnecessary, so I am providing a way to disable these.
Asserts
Some people prefer asserts instead of the sanity checks mentioned above. I.e.: If something is called with the wrong parameters the system should just assert that something is wrong and abort operation immediately. While I find working with a system that remains operation so that I can at least talk to it easier to work on, especially during bring up, I can accept that there are developers who prefer to assert.
There are two macros that need to be defined to get asserts working.
To enable asserts inside the JaeOS code itself define this:
#define RTOS_USE_ASSERTS
In order to actually have working assert the following macro needs to be defined:
#define RTOS_ASSERT(Cond)
The default provided in rtos.h will hang the system in an infinite loop if Cond is false. The system can only inspected using a debugger to find out why it has hung. This is suitable for some bare minimum embedded systems, but most application programmers probably would want something fancier and define their own implementation.
The implementation supposed to check if Cond holds true. If Cond is false the implementation should halt program execution. In most cases the implementation would also print some message regarding what went wrong and in which source file, function etc. Some assert macros can be quite elaborate and somewhat constitute an abuse of the C preprocessor. There is a reasonably mundane versions in the JaeOS examples included in the JaeOS package, I am not going to repeat it here. Even the mundane version has a rather elaborate nesting of C preprocessor macros.
Beyond the Bare Minimum
Running a minimalistic RTOS with just the bare minimum functionality certainly makes the system simple, but there one sometimes needs other functionality. The fall into two categories. Either optional features implemented inside the OS kernel, or utility packages that may be implemented on top of the actual OK kernel.
Extras
Extras are libraries implemented using the API functions of the RTOS, without having detailed knowledge how the RTOS works internally. At this time there is an implementation of message queues distributed with the JaeOS sources that is implemented as an extra (i.e. not part of the core OS).
Message Queues
Message queues are a common construct in most operating systems and many software packages written with a generic RTOS interface in mind require some support for message queues.
An implementation is provided with JaeOS. JaeOS message queues can be used both as a message FIFO or as a LIFO (stack). Messages are always read from the front of the queue, but functions are provided to append a message either at the end or at the front of the queue.
Messages that can be passed around are pointers to void, if the actual information fits into a pointer that can be passed around, otherwise the actual messages are expected to reside in some non-volatile area. Only the pointers are passed from task to task in the queue, the message bodies are expected to remain valid until the consumer of the message disposes of them.
The API consists of the functions listed below. They return RTOS_OK on success, RTOS_ERROR_OPERATION_NOT_PERMITTED if the queue parameter is NULL or if one tries to perform some operation on a queue that was not properly initialized by RTOS_CreateQueue() and RTOS_ERROR_FAILED if the operation cannot be completed.
RTOS_RegInt RTOS_CreateQueue(RTOS_Queue *queue, void *buffer, RTOS_QueueCount size);
Initialize a queue (turn a chunk of memory into a message queue structure).
The buffer area will be used to store pointers to messages, the size parameter is the number of items (void * pointers) that fir into the buffer.
RTOS_RegInt RTOS_DestroyQueue(RTOS_Queue *queue);
Make the queue uninitialized. Its buffer is "detached" from the queue and the size is set to 0 so as to make the queue no longer usable (so some task cannot post messages to it by accident).
This operation may be necessary if queues are generated by tasks that have a finite life time (such as client threads in a TCP/IP server) and it is necessary to ensure that once a queue is no longer in use some "background" thread can"t accidentally try to put something into its buffer since the buffer space might be reused.
The queue must be empty before it can be destroyed, of the queue is not empty RTOS_ERROR_FAILED is returned.
RTOS_RegInt RTOS_Enqueue(RTOS_Queue *queue, void *message, RTOS_Time timeout);
Append a message to the end of the queue. The timeout parameter works the same way as for RTOS_GetSemaphore(). It also means that RTOS_Enqueue() can be called from inside an ISR but only with timeout = 0.
RTOS_RegInt RTOS_PrependQueue(RTOS_Queue *queue, void *message, RTOS_Time timeout);
Works the same way as RTOS_Enqueue() but the message is placed to the front of the queue.
RTOS_RegInt RTOS_Dequeue(RTOS_Queue *queue, void **message, RTOS_Time timeout);
Consume one message from the front of the queue and return it to the caller in the message parameter. Timeout behaves the same as for RTOS_GetSemaphore(). This function may be called from an ISR but only with timeout=0.
RTOS_RegInt RTOS_PeekQueue(RTOS_Queue *queue, void **message);
Retrieve the first element in the queue and return it in message without actually removing it from the queue.
If the queue is empty RTOS_PeekQueue() returns RTOS_ERROR_FAILED.
Here is an example using message queues, ros_config.h:
#ifndef RTOS_CONFIG_H
#define RTOS_CONFIG_H
#define RTOS_INCLUDE_SEMAPHORES
#define RTOS_INCLUDE_DELAY
#define RTOS_TASK_NAME_LENGTH 32
#define RTOS_TICKS_PER_SECOND 100
#define RTOS_Priority_Producer 1
#define RTOS_Priority_Consumer1 2
#define RTOS_Priority_Consumer2 3
#define RTOS_Priority_Highest RTOS_Priority_Consumer2
#endif
And main.c:
#include <stdint.h>
#include <rtos.h>
#include <rtos_queue.h>
#include <board.h>
extern void PrintHex(uint32_t); // Print a number in HEX.
RTOS_StackItem_t stack1[512];
RTOS_StackItem_t stack2[512];
RTOS_StackItem_t stack3[512];
RTOS_StackItem_t stackIdle[RTOS_MIN_STACK_SIZE];
RTOS_Task task1;
RTOS_Task task2;
RTOS_Task task3;
RTOS_Task task_idle;
RTOS_Queue q;
void Consumer(void *p) // Take an item from the queue.
{
RTOS_RegInt res;
void *goodie;
while(1)
{
res = RTOS_Dequeue(&q, &goodie, RTOS_TIMEOUT_FOREVER);
if (RTOS_OK == res)
{
Board_Puts(RTOS_GetTaskName(RTOS_GetCurrentTask()));
Board_Putc(':');
PrintHex((uint32_t)goodie);
Board_Putc('\r');
Board_Putc('\n');
RTOS_YieldPriority();
}
}
(void)p;
}
void Producer(void *p) // Place an item into the queue.
{
uint32_t stuff = 1;
while(1)
{
stuff++;
Board_Puts(RTOS_GetTaskName(RTOS_GetCurrentTask()));
Board_Putc(':');
PrintHex(stuff);
Board_Putc('\r');
Board_Putc('\n');
RTOS_Enqueue(&q, (void *)(stuff), 0);
}
(void)p;
}
int main()
{
RTOS_CreateTask(&task1, RTOS_Priority_Producer, stack1, 512, &Producer, 0);
RTOS_SetTaskName(&task1, "Producer");
RTOS_CreateTask(&task2, RTOS_Priority_Consumer1, stack2, 512, &Consumer, 0);
RTOS_SetTaskName(&task2, "Consumer1");
RTOS_CreateTask(&task3, RTOS_Priority_Consumer2, stack3, 512, &Consumer, 0);
RTOS_SetTaskName(&task3, "Consumer2");
RTOS_CreateTask(&task_idle, RTOS_Priority_Idle,
stackIdle, RTOS_MIN_STACK_SIZE, &RTOS_DefaultIdleFunction, 0);
RTOS_SetTaskName(&task_idle, "Idle");
RTOS_CreateQueue(&q, buff, 16);
Board_HardwareInit();
RTOS_StartMultitasking();
Board_Puts("Something has gone seriously wrong!");
while(1);
return 0; // Unreachable.
}
Time Sharing
A strictly priority based scheduling algorithm is usually desirable if real time execution is needed. It is simple, predictable, easy to understand, and if something goes wrong it is usually quite obvious.
It is not possible however to describe in those terms that certain things are equally important and they all should progress.
The usual solution is to allow more than one task per priority and schedule tasks at the same priority level in some time sharing manner, usually using Round-Robin scheduling.
I did not want to give up the simplicity of the JaeOS scheduler design which does assume that priorities are unique. But in reality there really isn"t a need to actually run tasks at the same priority in order to have time sharing.
JaeOS has time sharing support, with the following interface:
To enable time sharing define this in rtos_config.h:
#define RTOS_SUPPORT_TIMESHARE
To create a task that is part of the time share create it with RTOS_CreateTimeShareTask():
RTOS_RegInt RTOS_CreateTimeShareTask(
RTOS_Task *task,
RTOS_TaskPriority priority,
void *sp0, unsigned long stackCapacity,
void (*f)(),
void *param,
RTOS_Time slice
);
The slice parameter defines the length of the task"s time slice or quantum. Different tasks do not need to have the time slices of the same length. E.g.: It is possible to describe that one task should be able to use the CPU on average twice as much as another task.
If the length of the time slice is defined as RTOS_TIMEOUT_FOREVER it effectively disables actual time slicing. It can be useful if some other properties of the time share tasks are desired without time slicing. In theory it is possible to create a cooperative multitasking system – possibly even without interrupts – using time sharing tasks with never expiring time slices.
While it sounds unlikely on modern hardware, I have actually worked with systems that did not even have interrupts (JaeOS did not exist at that time, so no I did not use there).
Time share tasks should be set up having adjacent priorities like so:
|
Priority |
0 |
1 |
2 |
3 |
4 |
5 |
6 |
|
Task |
Idle |
LowPrio |
TST1 |
TST2 |
TST3 |
TST4 |
HighPrio |
|
Notes |
Idle |
- |
Tasks Participating in the Time Share. |
|
|||
There is nothing in JaeOS to prevent the user to create some other pattern but the system behavior might not be what one considers reasonable, so stick to one block of time sharing tasks.
The JaeOS scheduler still behaves the same way as before when it comes to choosing a task to run. So LowPrio in this example will only run if none of the tasks in the time share are ready to run and HighPrio will run whenever it is ready to run even if there are tasks in the time share that are ready to run.
If more than one tasks in the time share happen to be ready to run the higher priority will be chosen by the JaeOS scheduler.
What is different is that each task in the time share will have its time quantum decremented by OS at each timer tick when it is running. The quantum is decremented whenever the scheduler is run unless it has already been decremented during the same tick (to account for tasks switching faster than the timer interrupts happen). Thus it is possible to decrement the quantum of more than one in the time share tasks during the same timer tick if they happen to be running at one point or another during the same tick. Measuring time with finer resolution than the timer tick would be desirable, but in practice the above is sufficient.
When a tasks time quantum reaches zero it is removed from the ready to run set and added to a pre-empted set. It is also added to the end of a doubly linked list pre-empted tasks.
The JaeOS also checks if at least one task in the time share is ready to run, and if none of the time share tasks are ready to run at the time it will remove the first pre-empted task from the doubly linked list and place it into the ready to run set with a new time quantum.
Thus if all time share tasks just do calculations burning processor cycles they will be rotated in a Roubd-Robin fashion.
There is one additional change in behavior, regarding synchronization such as acquiring semaphores or waiting for events (which is internally the same code).
Remember that normally tasks have a chance to get a semaphore in the order of their priorities. I.e. It is always the highest priority task that will get a semaphore even if the lower priority ones have been waiting for it a lot longer.
This is still true for tasks outside the time share, however if the highest priority task waiting for the semaphore is in the time share it might not get the semaphore. There is a doubly-linked list of all time share tasks waiting for the semaphore and the first one on the list is the one that gets the semaphore.
In other words the tasks in the time share when interacting with semaphores behave as if they all had the same priority and get the semaphore first come first served (the one which started waiting first gets the semaphore first).
The time quantum of a task is not reset when it starts to wait for a semaphore or goes to sleep by calling RTOS_Delay(), when it is awakened later on it continues to run with what is left from its old time quantum until it counts down and the task goes to the pre-empted list. Thus a task cannot replenish its time quantum by sleeping nor is it specifically penalized to it.
After acquiring a semaphore or waking up from sleep a task is placed into the ready to run set. This might result in situations when more than one task in the time share is ready to run at the same time. JaeOS just runs the scheduler as normal, so always the one with the highest priority is allowed to actually run. Eventually its time quantum will expire and the task is placed at the end of the pre-empted list. Since there are other tasks in the time share that are ready to run no task from the pre-empted list is made ready to run until they all burn their time quantum and get pre-empted.
Thus any advantage of having a higher priority within the time share block is only temporary, eventually pre-emption because if time quantum expiration determines how much CPU time each task gets.
If a task specifically wishes to give up its time quantum before it reaches zero it can do so by calling RTOS_YieldTimeSlice(). This call places the task at the end of the pre-empted list as it its time quantum has expired.
Symmetric Multiprocessing (SMP)
Some systems (even systems on a chip) have more than one CPU cores. It is sometimes logical to use all of them or at least more than one of them to execute tasks actually in parallel – that is not one CPU switching between tasks very quickly but more than one CPU cores executing different tasks actually at the same time.
JaeOS has some rudimentary support to allow running tasks on more than one CPU core within the same copy of JaeOS (i.e. not running a separate incarnation of JaeOS on each CPU core, but running one copy of JaeOS that schedules tasks on more than one CPU cores).
This feature is still considered to be somewhat experimental because it only has been tested on the two ARM cores inside a Zynq processor in a ZedBoard, but the results are somewhat encouraging. I was able to run a slightly hacked version of the JaeOS/ + EFCI "telnet demo" using both CPU cores. The most serious issue is that in order to do that I had to disable data cache so that data in RAM looks the same to the two CPU cores as well as to DMA. The two CPU cores can be configured to see the changes made by the other core, but simply invalidating pieces of the data cache on one core after a DMA operation does not seem to make the data equally visible on the other core. Perhaps there is some way around it, but so far I was only able to make it work with the data cache off.
For this reason and because the SMP features were not reviewed to the same detail as the rest of the code I consider SMP support in JaeOS to be experimental, even though being able to run something as complex as a TCP/IP stack with telnet servicing multiple clients is encouraging.
In the simplest case, if the application is written correctly, running on multiple CPU cores is a lot like running on a single core with the added benefit of actually executing multiple tasks at the same time.
Only some targets support SMP, among other things, the target system actually has to have more than one CPU cores to be able to do SMP at all.
In rtos_config.h define the following macros:
#define RTOS_SMP /* Compile in SMP support. */
#define RTOS_SMP_CPU_CORES 2 /* The number of cores assigned to JaeOS to use. */
You must define the number of CPU cores to be used by JaeOS and it has to be at least 1. Of course running SMP on 1 core only makes sense for testing / bring up purposes, one normally would want to use 2 or more cores in SMP mode.
Critical Sections in SMP Mode
During SMP operation, apart from disabling interrupts on the local core, JaeOS uses a global lock variable that has to be acquired when a CPU enters a critical section, including at the beginning of an ISR (at least in the existing target). If the lock is used by a different CPU the core has to wait until it is released. The implementation details are architecture specific, but modern processors usually have instructions to support this locking efficiently. This makes critical sections exclusive globally, on all CPU cores in the system assigned to JaeOS. It also makes it impossible to service interrupts on any core while any code anywhere is executing in a critical section.
Again, at least in theory, this could be finer grained depending on the target systems capabilities and needs, but the entire SMP support in JaeOS is currently experimental and no analysis has been performed how to fine tune it for multiprocessing (and if you really need fine tuned and optimized performance on a multiprocessing platform you might need a bigger OS such as Linux).
Time Sharing and SMP
If Time Shares are enabled the following macro may be used to define how many time shared tasks are allowed to run in parallel.
#define RTOS_TIMESHARE_PARALLEL_MAX 1 /* It has to be between 1 – number of cores. */
If it is not defined explicitly it defaults to RTOS_SMP_CPU_CORES . This is used to initialize an internal variable in JaeOS which in turn is used by the Time Share code when it decides to make some previously preempted tasks in the Time Share ready to run. On single processor systems we make sure that at least one task participating in the Time Share is ready to run, but if there are multiple cores we may be able to run more than one at a time. The reason why it is configurable (and it isn"t hardcoded to e.g. all cores) is that JaeOS allows restricting which tasks can run on which cores. Thus if e.g. all tasks in the Time Share are restricted to only run on e.g. the first core (or only on the second, whatever the case may be) it is possible that we can only run some fewer number of them at the same time than the actual number of cores in the system.
Restricting Tasks to Certain CPUs
There is a system call that actually tells JaeOS which task is allowed to run on which CPU core (default is that a task can run on all CPUs):
RTOS_RegInt RTOS_RestrictTaskToCpus(RTOS_Task *task, RTOS_CpuMask cpus);
Task must be an existing task (already initialized by RTOS_CreateTask()). The actual cores are passed to the function as a bitmap (1 = first CPU, 2 = second CPU, 3 = First and second CPU, etc.).
It is useful if the application programmer wants to split up work between CPU cores and make each one perform different functions. It can be necessary for various reasons, e.g. it might be easier to deal with certain peripherals (especially ones that access system memory via DMA) from one CPU.
Also, e.g. the SMP implementation that I have been using on the ZedBoard always services interrupts from peripherals on the first CPU core. Which means the other core doesn"t really need to run task level code when it is idling. Thus only one Idle task is created which is then restricted to the first core.
The second core is simply sitting in a "holding pen" inside the ISR code when it has no useful work to do – it is woken up by events sent from the first core when it is time to re-run the scheduler.
Scheduling in SMP Mode
For SMP operation the scheduler code has been modified to not pick a task that is already running on another core, but otherwise it works just like the single CPU scheduler. While JaeOS supports multiple CPU cores executing tasks in parallel it does not provide a centralized scheduler. I.e.: Even in SMP mode there is no centralized scheduler, each core runs the scheduler independently and picks up a task from those that are ready to run, the only information regarding what is going on elsewhere in the system is if a task is actually already executing on another core.
When something of interest happens, in particular when the set of ready to run tasks changes, CPU cores raise events and generate software interrupts on the other cores so that the scheduler is re-run on those.
This mechanism allows CPUs to pick up work when work is available independent of each other. In theory it should scale well to systems with somewhat larger number of CPUs.
A lack of central decision making has some drawbacks however. In particular each CPU will pick up the highest priority work when at the time it is running the scheduler. There is no reshuffling of work between CPU cores even if migrating tasks to other cores would end up with higher priority tasks being executed.
E.g. Let us assume that there are 3 tasks in a system and Task1 and Task3 can run on either core and Task2 is restricted to the first core, and theirs priorities are 1, 3, and 2 respectively.
|
Task Name |
Task1 |
Task2 |
Task3 |
|
Task Priority |
1 |
2 |
3 |
|
CPUs Allowed |
0 and 1 |
0 |
0 and 1 |
In one scenario they might end up in a situation that the first core is executing Task3, the second core is executing Task1 and Task2 is ready to run but it does not get scheduled. This is because the first core that could execute Task2 is occupied by a higher priority task (Task3) and the second core that is running a lower priority task (Task1) cannot run Task2.
|
Task |
Task1 |
Task2 |
Task3 |
|
CPU |
1 |
Waiting for a CPU. |
0 |
Of course depending on how the situation has developed till the present, we can end up with a scenario where the second core is running Task3 and the first core is running Task2 which seems to be more desirable since higher priority tasks are running.
|
Task |
Task1 |
Task2 |
Task3 |
|
CPU |
Waiting for a CPU. |
0 |
1 |
But JaeOS does not have any mechanism to go from the first situation to the second, since it lacks any sort of centralized decision making regarding which task is assigned to which CPU core. So either the first or the second scenario can happen (depending on when each task becomes ready to run and gets scheduled on an available CPU) but JaeOS does not have any notion that the second situation is better than the first or how to go from the first to the second.
Note: While this is not ideal, the SMP system is still doing better than a single CPU system would. Apart from the highest priority task one additional task is progressing at the same time.
At this time there are no plans to implement any centralized scheduling in JaeOS, nor the code needed to force a task to migrate to a different CPU.
Also note that the example is somewhat artificial. Tasks should only be restricted to run on certain CPUs for a good reason.
If all tasks are allowed to run on all cores then the highest priority tasks will be picked up by the available cores.
|
Task Name |
Task1 |
Task2 |
Task3 |
|
Task Priority |
1 |
2 |
3 |
|
CPUs Allowed |
0 and 1 |
0 and 1 |
0 and 1 |
If some compartmentalization is needed one can restrict each task to only run on one core or in general have distinct groups of tasks assigned to distinct CPU cores. The state of the system will be much better predictable. E.g. The following set up behaves in a predictable way because each task can only run on one of the CPU cores.
|
Task Name |
Task1 |
Task2 |
Task3 |
Task4 |
|
Task Priority |
1 |
2 |
3 |
4 |
|
CPUs Allowed |
0 |
1 |
0 |
1 |
JaeOS® is a registered trademark of Andras Zsoter in Canada.