I would like to write a function (in hlasm) which calls itself and other functions.
On a x86 or z80 (and probably others) you would do just call function and at the end of it a ret. The processor will then store and retrieve the return address.
The instruction set has the following instruction:
BAL reg,func
...which stores the return address in reg, then at the end you can do BR regto jump to that return address. Another problem though is that there are apparently no push/pop instructions.
In System/360 and successor operating systems, the infrastructure to do this is a part of what is called reentrant programming. The basic concept is that the storage for the save area pointed to by R13 is obtained from the operating system (think malloc in C). The system macro calls to obtain storage are used at the very beginning of your program. Likewise, the system macro calls to return the storage to the operating system are coded at the exit of your program
You haven't mentioned which operating system you are using. I am going to make the following assumptions for this example code:
This skeletal routine is reentrant, and because of that it can also be used recursively, as you mentioned (with caveats that bad code reinvoking itself will eventually lead to out-of-storage abends). The code requires a base register, but I suggest using the modern coding method called "baseless", where the code itself does not use a base register, as it uses the relative & immediate instructions for branches. (You always need a base register for data.)
WORKAREA DSECT , Reentrant work area (like C stack)
DS 18F Save area
FIELD1 DS F Some variable
FIELD2 DS F Another variable
WORKLEN EQU *-WORKAREA Length of reentrant work area
SUBRTN1 RSECT , HLASM will perform reentrant checking
STM R14,R12,12(R13) Save registers at entry
LR R12,R15 Set code base register
USING SUBRTN1,R12 Establish code addressability
LGHI R0,WORKLEN Get length of reentrant work area
STORAGE OBTAIN, Obtain reentrant work area X
LENGTH=(0) ..Length is in R0
ST R1,8(,R13) Forward chain in prev save area
ST R13,4(,R1) Backward chain in next save area
L R14,20(,R13) Get R1 at entry (parameters)
LR R13,R1 Set up new save area/reentrant workarea
USING WORKAREA,R13 Establish work area addressability
LM R2,R3,0(R14) Get addresses of parameters
STM R2,R3,FIELD1 Save parameter addresses for later
…
*** Logic goes here
…
LR R1,R13 Address to be released
L R13,4(,R13) Address of prior save area
LGHI R0,WORKLEN Length of storage to release
STORAGE RELEASE, Release reentrant work area X
ADDRESS=(1), ..Address in R1 X
LENGTH=(0) ..Length in R0
LM R14,R12,12(R13) Restore registers
OI 15(R13),X'01' This bit on means this save area is inactive
BR R14 Return to caller
This is a very basic example, and introduces some advanced assembler concepts, like a DSECT (dummy section), which describes an area without actually allocating storage in the program. RSECT is a way for the assembler to reinforce reentrancy, by issuing warnings if the program tries to modify itself. (There's also an assembler option, RENT, but that does it for the entire source; RSECT is just for that section.)
The key thing to remember about this example is that you have two base registers, one addressing code, one addressing data. This has parallels with the early x86 architecture, with the code segment and data segment. In this case, the data segment also is used as the stack segment.
Reentrant programs is an essential part of systems-level z/Architecture programming, as it's practically required for multitasking environments. It's an important technique to understand if one will code applications (in a general sense) beyond typical batch programs.
If you are in an LE environment (COBOL, PL/I, C/C++, or assembler main program), you should use the LE-supplied CEEENTRY and CEETERM macros. LE-compiler-generated programs are generally reentrant (even COBOL with the NORENT option). Make sure you code MAIN=NO on the CEEENTRY macro, otherwise all sorts of performance problems and possible logic errors occur. The LE stack mechanism is an advanced version of the technique that I demonstrated above, most notably allocating a large area dedicated to stack usage, so that subsequent calls don't have the overhead of calling the operating system to obtain storage.
If you are working in a z/VSE or z/VM environment, or the non-IBM BS2000/OSD environment, I can modify the above example.
Please comment with questions, and I will update this example for additional clarity.