Use of SRAM0 and SRAM1 in Linux kernel driver module

Discussion around products based on ARM Cortex-A5 core.

Moderator: nferre

mwi
Posts: 25
Joined: Wed Jul 30, 2014 9:17 pm

Use of SRAM0 and SRAM1 in Linux kernel driver module

Tue Sep 16, 2014 12:11 pm

I want to use the internal SRAM in my kernel driver module (high speed data exchange with FIQ).

I searched the Linux 3.16 sources for uses of SAMA5D3_SRAM_BASE. I did not find any use except of the initialisation during boot.

Does anyone know, if the internal SRAM is used by Linux on SAMA5D3X or if it is save to use the memory in my kernel driver module?
mwi
Posts: 25
Joined: Wed Jul 30, 2014 9:17 pm

Re: Use of SRAM0 and SRAM1 in Linux kernel driver module

Wed Sep 17, 2014 10:04 am

moved the data structure to the SRAM with no effect (speed up of FIQ handler).

Did a test with the following code:

Code: Select all

        ldr     r11, [r8, #0x0008]
        movw    r12, #0x0000
        movt    r12, #0x4000
        str     r12, [r11, #0x0030]
        str     r12, [r11, #0x0034]

        ldr     r9, [r8, #0x0008]
        ldr     r9, [r8, #0x0008]
        ldr     r9, [r8, #0x0008]
        ldr     r9, [r8, #0x0008]
        ldr     r9, [r8, #0x0008]
        ldr     r9, [r8, #0x0008]
        ldr     r9, [r8, #0x0008]
        ldr     r9, [r8, #0x0008]
        ldr     r9, [r8, #0x0008]
        ldr     r9, [r8, #0x0008]
        ldr     r9, [r8, #0x0008]
        ldr     r9, [r8, #0x0008]
        ldr     r9, [r8, #0x0008]
        ldr     r9, [r8, #0x0008]
        ldr     r9, [r8, #0x0008]
        ldr     r9, [r8, #0x0008]
        ldr     r9, [r8, #0x0008]
        ldr     r9, [r8, #0x0008]
        ldr     r9, [r8, #0x0008]
        ldr     r9, [r8, #0x0008]

        ldr     r11, [r8, #0x0008]
        movw    r12, #0x0000
        movt    r12, #0x4000
        str     r12, [r11, #0x0030]
        str     r12, [r11, #0x0034]

        and     r13, r11, r12
        and     r13, r11, r12
        and     r13, r11, r12
        and     r13, r11, r12
        and     r13, r11, r12
        and     r13, r11, r12
        and     r13, r11, r12
        and     r13, r11, r12
        and     r13, r11, r12
        and     r13, r11, r12
        and     r13, r11, r12
        and     r13, r11, r12
        and     r13, r11, r12
        and     r13, r11, r12
        and     r13, r11, r12
        and     r13, r11, r12
        and     r13, r11, r12
        and     r13, r11, r12
        and     r13, r11, r12
        and     r13, r11, r12

        ldr     r11, [r8, #0x0008]
        movw    r12, #0x0000
        movt    r12, #0x4000
        str     r12, [r11, #0x0030]
        str     r12, [r11, #0x0034]
Toggling GPIO before each block of instructions, the ones just ANDing registers (no DRAM access), the other ones loading values from DRAM.

On scope I can see that:
  • both instruction blocks take almost the same time (would have exspected that loading from DRAM takes much longer time than just working on registers)
  • Instruction blocks (both) take much longer time than they should (>= 200ns). Assuming the 536MHz this should be enough for approx. 120 instructions (Especially of the ones only working on registers). But only 20 are executed (Assuming that inside FIQ handler nothing else is done).
  • The execution time of the instructions differs in length over time. I would not expect this on assembler instructions.
The only reason I can see is that the FIQ handler itself is stored and executed in DRAM and therefore rather slow in execution (depends on cache, ...).

Therefore I want to move the FIQ handler function (only Assembly) into SRAM0. Does anyone know how to do? As far as I know the function is now loaded to the end of the IRQ Vector table by use of set_fiq_handler (...).
mwi
Posts: 25
Joined: Wed Jul 30, 2014 9:17 pm

Re: Use of SRAM0 and SRAM1 in Linux kernel driver module

Wed Sep 17, 2014 11:15 am

I checked the PC before and after the test instruction block.

It incremented by 224. If i remember correctly, this gives 224/4=56 instructions as every instruction is 32Bit on ARM.

This fits to the number of instructions done between the 2 samplings of PC. Still the execution time is much too slow (approx. 880ns - should be time for >400 instructions).

PC points to 0xFFFF123C at beginning of test instruction block. Taking a look at the data sheet this would be inside "reserved" block of "Internal Peripherals" which very likely is wrong. So the PC seems to work on a VA.

Please let me know if I am completely on the wrong way at some of the points.
blue_z
Location: USA
Posts: 1500
Joined: Thu Apr 19, 2007 10:15 pm

Re: Use of SRAM0 and SRAM1 in Linux kernel driver module

Wed Sep 17, 2014 8:53 pm

mwi wrote:both instruction blocks take almost the same time (would have exspected that loading from DRAM takes much longer time than just working on registers)
It's been a while since I've dealt with instruction execution times, but your assumptions seems to be based on CISC rather than RISC.

There have been a few kernel patches (that I'm aware of, i.e. patches for macb.c and atmel_nand.c) that utilized the SRAM. But they used the SRAM for data buffers.
mwi wrote:The execution time of the instructions differs in length over time.
I would not expect this on assembler instructions.
Huh?
Double huh?

Regards
mwi
Posts: 25
Joined: Wed Jul 30, 2014 9:17 pm

Re: Use of SRAM0 and SRAM1 in Linux kernel driver module

Fri Sep 19, 2014 10:09 am

@blue_z: do you mean the execution time is the same, if working from DDR RAM or SRAM? So I would not have any advantage if woring from SRAM? Still the execution time is not constant (always the same). I can see the jitter on scope between the 3 toggles of GPIO.

I tried to start the FIQ handler code from SRAM - copied it there and checked back with JTAG debugger and SAM-BA. It definitely is located in SRAM at physcical address 0x00300100. I check the function in Linux every 2 seconds for any corruption - it stays OK over run-time of the Linux system. So I am pretty sure that the handler does not get corrupted in SRAM by and other task.

Now I changed the vector address for the FIQ in AIC_SVR to point to the virtual address at SRAM, but unfortunately the FIQ handler is still called from its old position (tested this by replacing FIQ handler with all 0x00000000 - invalid Ops in SRAM, but FIQ handler continued to work normally).

Any ideas, why the AIC does not branch to the FIQ vector address if a FIQ occurs? Is Linux changing the AIC configuration somewhere?

Afterwards I tried to branch to the SRAM location of the FIQ handler by doing the following:

Code: Select all

        /* load VA of FIQ handler location in SRAM0 and branch there */
        ldr     r10, [r8, #0x0040]

        /* skip the "branch to SRAM" commands after branch to SRAM0 */
        add     r10, #12

        /* branch to SRAM */
        mov     pc, r10
But after branch I always get a kernel panic:

Code: Select all

Bad mode in prefetch abort handler detected
Internal error: Oops - bad mode: 0 [#1] PREEMPT ARM
Modules linked in: adu_adb_spi_driver_hw(O) spi_atmel_mtx(O)
CPU: 0 PID: 0 Comm: swapper Tainted: G           O  3.16.0 #12
task: c05a4730 ti: c059a000 task.ti: c059a000
PC is at 0xe093810c
LR is at arm_cpuidle_simple_enter+0xc/0x14
pc : [<e093810c>]    lr : [<c00121cc>]    psr: 600f01d1
sp : c059bf38  ip : 00000019  fp : 00000000
r10: 00000000  r9 : c059a000  r8 : c05c95cf
r7 : c05c03cc  r6 : c05bff28  r5 : 0000004b  r4 : 00000000
r3 : c00121c0  r2 : 00000000  r1 : c05c03cc  r0 : c05bff28
Flags: nZCv  IRQs off  FIQs off  Mode FIQ_32  ISA ARM  Segment kernel
Control: 10c53c7d  Table: 3f65c059  DAC: 00000015
Process swapper (pid: 0, stack limit = 0xc059a238)
Stack: (0xc059bf38 to 0xc059c000)
bf20:                                                       c05bff28 c05c03cc
bf40: 00000000 c00121c0 00000000 0000004b c05bff28 c05c03cc c05c95cf c059a000
bf60: 00000000 00000000 00000019 c059bf38 c00121cc e093810c 600f01d1 ffffffff
bf80: 7c6cb9db c02ff008 7c6cb9db 0000004b c05a2080 c05c03cc c05bff28 c05a2080
bfa0: c05c03cc c05bff20 c05c95cf c0044b14 c05a4730 c0591890 00000000 c056bb80
bfc0: ffffffff ffffffff c056b5f4 00000000 00000000 c0591890 00000000 c05c9914
bfe0: c05a2020 c059188c c05a57e0 20004059 410fc051 20008070 00000000 00000000
[<c00121cc>] (arm_cpuidle_simple_enter) from [<c05c03cc>] (0xc05c03cc)
Code: e3a00000 e598a040 e28aa00c e1a0f00a (e598b008) 
---[ end trace e919ca56b527c040 ]---
Kernel panic - not syncing: Attempted to kill the idle task!
---[ end Kernel panic - not syncing: Attempted to kill the idle task!
PC exactly points to the correct virtual address that I get for the SRAM FIQ handler position, so he seems to branch to the correct address. Still I get the panic. I think I am doing something bad here, but do not know what this is. Any ideas?

What makes me wonder a bit is that he is not in FIQ_32 mode, but PREEMP_ARM. Should still be in FIQ mode after branch, correct?
mwi
Posts: 25
Joined: Wed Jul 30, 2014 9:17 pm

Re: Use of SRAM0 and SRAM1 in Linux kernel driver module

Fri Sep 19, 2014 10:33 am

just read in the datasheet that the AIC IRQ vectoring is not used in most cases a OS is used, as they implement their own IRQ handling. Is this true for the 3.16 AT91 kernel, too (likely yes, as this would explain why changing the vector address does not make any difference)?

Additionally this would explain why the time from falling edge of the Data Ready of my ADC (FIQ source) to first toggle of GPIO inside my FIQ handler is not constant. If AIC is used with no software in between, this should always be the same time.

Follow Up:
Googled a bit and seems that AIC driver for SAMA5D3x will be part of next kernel release. Did not find the AIC driver in my 3.16 sources at all (also not in dmesg bootup messages).

I will give 3.17-rc5 a try and check, if the AIC is supported there for SAMA5d3x
blue_z
Location: USA
Posts: 1500
Joined: Thu Apr 19, 2007 10:15 pm

Re: Use of SRAM0 and SRAM1 in Linux kernel driver module

Fri Sep 19, 2014 8:06 pm

mwi wrote:I think I am doing something bad here, but do not know what this is. Any ideas?
What method did you use to map the SRAM into kernel space and give those pages execute privilege?

Regards
mwi
Posts: 25
Joined: Wed Jul 30, 2014 9:17 pm

Re: Use of SRAM0 and SRAM1 in Linux kernel driver module

Mon Sep 22, 2014 8:43 am

just copied the sources to SRAM addresses and tried to set the Source Vector Address of AIC to branch there - did not work, because AIC is not implemented?

in second approach directly branched to the SRAM from Assembler, where I get the kernel panic.

did I miss to configure something in SRAM? I know I do. Any comment is appreciated. How can I set the SRAM to be "executeable" (code)?
blue_z
Location: USA
Posts: 1500
Joined: Thu Apr 19, 2007 10:15 pm

Re: Use of SRAM0 and SRAM1 in Linux kernel driver module

Mon Sep 22, 2014 9:47 pm

mwi wrote:in second approach directly branched to the SRAM from Assembler, where I get the kernel panic.
Branch to what address?
You would need a virtual address; you cannot branch to the physical address of the SRAM once the kernel enables the MMU for virtual memory (during system initialization).
mwi wrote:did I miss to configure something in SRAM?
If you have to ask, then you probably did.
Normal memory is handed over to the kernel for management through the Device Tree (or old-style ATAGs) or the mem parameter in the kernel command line.
But once you hand over "special" memory such as SRAM to the memory pool, there is no way for your driver to allocate or reserve it.
The typical method of providing a driver with reserved memory is to pass the memory region's location and size to the driver (Device Tree makes this very easy compared to ATAGs) while excluding this memory region from the kernel's memory pool.
Then the driver can claim the memory region with request_mem_region() and then get a virtual memory mapping with ioremap().
mwi wrote:How can I set the SRAM to be "executeable" (code)?
I don't have an answer for that. It's done for a user process when a program is loaded.
But I don't expect to find a kernel interface for a driver to make a page executable; that sounds like a security hole.

The MMU has attribute bits for each page of virtual memory.
But the software interface is through virtual-memory-mapping structures. VM_EXEC is the attribute that allows execution of code (aka text) in that page of memory. Take a look at how install_special_mapping() is called in this ARM code.

Note that is easier for a user program to make the SRAM executable than a kernel driver. The SRAM can be specified using /dev/mem and then mmap()'d with the PROT_EXEC capability.

Regards
mwi
Posts: 25
Joined: Wed Jul 30, 2014 9:17 pm

Re: Use of SRAM0 and SRAM1 in Linux kernel driver module

Tue Sep 23, 2014 10:24 am

Branch to what address?
VA I got from ioremap for SRAM at 0x00300000.
J4x
Posts: 11
Joined: Thu Jun 22, 2017 8:00 pm

Re: Use of SRAM0 and SRAM1 in Linux kernel driver module

Fri Jun 30, 2017 3:49 pm

Hello @mwi.

It's passed quite a few years since this thread was last updated but I was looking around on how to implement FIQ and came across it.
  • Did you manage to get your FIQ working?
  • Did you get to use SRAM with your driver?
  • Did you put any memory region in your driver's entry in device-tree?
Thanks!

Return to “SAMA5D Cortex-A5 MPU”

Who is online

Users browsing this forum: No registered users and 1 guest