summaryrefslogtreecommitdiff
path: root/lib/rsa
AgeCommit message (Collapse)Author
2023-01-11lib: rsa: cosmetic: fix building warningHaijun Qin
add initialization of variable 'node',this can aviod the building warning: 'node' may be used uninitialized [-Wmaybe-uninitialized] Signed-off-by: Haijun Qin <qinhaijun@eswincomputing.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2022-12-23rsa-verify: Rework host check for CONFIG_RSA_VERIFY_WITH_PKEYTom Rini
While we do not want to use CONFIG_RSA_VERIFY_WITH_PKEY on the host, we cannot undef the symbol in this manner. As this ends up being a test within another function we can use !tools_build() as a test here. Cc: AKASHI Takahiro <takahiro.akashi@linaro.org> Cc: Simon Glass <sjg@chromium.org> Signed-off-by: Tom Rini <trini@konsulko.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2022-09-15lib: rsa: fix padding_pss_verifyHeinrich Schuchardt
Check the that the hash length is shorter than the message length. This avoids: ./tools/../lib/rsa/rsa-verify.c:275:11: warning: ‘*db’ may be used uninitialized [-Wmaybe-uninitialized] 275 | db[0] &= 0xff >> leftmost_bits; Fixes: 061daa0b61f0 ("rsa: add support of padding pss") Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2022-07-07spl: Ensure all SPL symbols in Kconfig have some SPL dependencyTom Rini
Tighten up symbol dependencies in a number of places. Ensure that a SPL specific option has at least a direct dependency on SPL. In places where it's clear that we depend on something more specific, use that dependency instead. This means in a very small number of places we can drop redundant dependencies. Reported-by: Pali Rohár <pali@kernel.org> Signed-off-by: Tom Rini <trini@konsulko.com>
2022-04-11lib: rsa: Update function padding_pss_verify (any-salt)SESA644425
Modify function to support any salt length instead of max length only. Function now detects salt length by parsing the content of db buffer. Note that it works with (but is not limited to) zero-length, digest-length and max-length Signed-off-by: SESA644425 <gioja.hermann@non.se.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2022-04-11lib: rsa: Leverage existing data buffer instead of systematic copySESA644425
Prior to introduction of modifications in rsassa_pss functions related to padding verification, doing a pass to reduce memory consumption of function by replacing memory copies of parts of const buffer by pointers to the original buffer (masked_db and h are subparts of msg buffer which is declared const, salt is a subpart of db which is a working buffer, unmodified after being filled). New pointers scope is limited to the function where they are declared (not returned to caller by any mean), zeroing risk of memory fault related to the change. Signed-off-by: SESA644425 <gioja.hermann@non.se.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2022-04-11lib: rsa: Fix const-correctness of rsassa_pss functionsSESA644425
Prior to introduction of modifications in rsassa_pss functions related to padding verification, doing a pass to update const-correctness in targeted functions to comply with coding-rules and avoid const-cast Signed-off-by: SESA644425 <gioja.hermann@non.se.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2022-03-31lib: rsa: allow rsa verify with pkey in SPLPhilippe Reynes
This commit adds the option SPL_RSA_VERIFY_WITH_PKEY. Reviewed-by: Simon Glass <sjg@chromium.org> Signed-off-by: Philippe Reynes <philippe.reynes@softathome.com>
2022-03-04lib: rsa: use actual OpenSSL 1.1.0 EVP MD APIYann Droneaud
Since OpenSSL 1.1.0, EVP_MD_CTX_create() is EVP_MD_CTX_new() EVP_MD_CTX_destroy() is EVP_MD_CTX_free() EVP_MD_CTX_init() is EVP_MD_CTX_reset() As there's no need to reset a newly created EVP_MD_CTX, moreover EVP_DigestSignInit() does the reset, thus call to EVP_MD_CTX_init() can be dropped. As there's no need to reset an EVP_MD_CTX before it's destroyed, as it will be reset by EVP_MD_CTX_free(), call to EVP_MD_CTX_reset() is not needed and can be dropped. Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>
2022-01-28rsa: adds rsa3072 algorithmJamin Lin
Add to support rsa 3072 bits algorithm in tools for image sign at host side and adds rsa 3072 bits verification in the image binary. Add test case in vboot for sha384 with rsa3072 algorithm testing. Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2022-01-26image: Return destination node for add_verify_data() methodSimon Glass
It is useful to know where the verification data was written. Update the API to return this. Signed-off-by: Simon Glass <sjg@chromium.org>
2022-01-26rsa: Add debugging for failure casesSimon Glass
Add some more debugging to make it easier to see what is being tried and what fails. Fix a few comment styles while here. Signed-off-by: Simon Glass <sjg@chromium.org>
2022-01-19doc: replace @return by Return:Heinrich Schuchardt
Sphinx expects Return: and not @return to indicate a return value. find . -name '*.c' -exec \ sed -i 's/^\(\s\)\*\(\s*\)@return\(\s\)/\1*\2Return:\3/' {} \; find . -name '*.h' -exec \ sed -i 's/^\(\s\)\*\(\s*\)@return\(\s\)/\1*\2Return:\3/' {} \; Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
2022-01-10Merge branch 'next'Tom Rini
Signed-off-by: Tom Rini <trini@konsulko.com>
2022-01-10lib/rsa: avoid -Wdiscarded-qualifiersHeinrich Schuchardt
The return type of EVP_PKEY_get0_RSA() is const struct rsa_st *. Our code drops the const qualifier leading to In file included from tools/lib/rsa/rsa-sign.c:1: ./tools/../lib/rsa/rsa-sign.c: In function ‘rsa_add_verify_data’: ./tools/../lib/rsa/rsa-sign.c:631:13: warning: assignment discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers] 631 | rsa = EVP_PKEY_get0_RSA(pkey); | ^ Add a type conversion. Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
2022-01-09Merge tag 'u-boot-amlogic-20220107' of ↵Tom Rini
https://source.denx.de/u-boot/custodians/u-boot-amlogic into next - disable CONFIG_NET_RANDOM_ETHADDR when unnecessary on amlogic based configs - meson64_android: add board specific env settings, in order to support VIM3/L for android - add changes to support VIM3/L android boot by using meson64_android.h config
2021-12-26tools: avoid OpenSSL deprecation warningsHeinrich Schuchardt
Our Gitlab CI buildsystem is set up to treat warnings as errors. With OpenSSL 3.0 a lot of deprecation warnings occur. With the patch compatibility with OpenSSL 1.1.1 is declared. In the long run we should upgrade our code to use the current API. A -Wdiscarded-qualifiers warning is muted by casting. Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
2021-11-17crypto: aspeed: Add AST2600 ACRY supportChia-Wei Wang
ACRY is designed to accelerate ECC/RSA digital signature generation and verification. Signed-off-by: Chia-Wei Wang <chiawei_wang@aspeedtech.com>
2021-10-26lib: rsa: rsa-verify: also check that padding is not NULLPhilippe Reynes
This commit adds a check on the padding in the function rsa_verify_key to avoid using a NULL pointer. Signed-off-by: Philippe Reynes <philippe.reynes@softathome.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2021-10-20Merge tag 'u-boot-imx-20211020' of ↵Tom Rini
https://source.denx.de/u-boot/custodians/u-boot-imx u-boot-imx-20211020 ------------------- First PR from u-boot-imx for 2022.01 CI: https://source.denx.de/u-boot/custodians/u-boot-imx/-/pipelines/9535 - new board: kontron-sl-mx8mm - imx8m: - fix secure boot - imx ESDHC: fixes - i.MX53: Support thum2, bmode and fixes for Menlo board usbarmory switch to Ethernet driver model - imx6 : - DDR calibration for Toradex boards - imx7: - Fixes - Updated gateworks boards (ventana / venice) # gpg verification failed.
2021-10-08image: Use Kconfig to enable FIT_RSASSA_PSS on hostSimon Glass
Add a host Kconfig for FIT_RSASSA_PSS. With this we can use CONFIG_IS_ENABLED(FIT_RSASSA_PSS) directly in the host build, so drop the forcing of this in the image.h header. Drop the #ifdef around padding_pss_verify() too since it is not needed. Use the compiler to check the config where possible, instead of the preprocessor. Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Alexandru Gagniuc <mr.nuke.me@gmail.com>
2021-10-07Kconfig: Don't use RSA_FREESCALE_EXP on MX7ULPRicardo Salveti
The CAAM on IMX7ULP doesn't support public key hardware acceleration (PKHA), as in other NXP parts. Disable RSA_FREESCALE_EXP for IMX7ULP too. Fixed: f4e9ff7135 ("Kconfig: Don't use RSA_FREESCALE_EXP on IMX") Signed-off-by: Ricardo Salveti <ricardo@foundries.io> Signed-off-by: Oleksandr Suvorov <oleksandr.suvorov@foundries.io>
2021-10-06lib: rsa: fix dependency for SPL_RSA_VERIFYOleksandr Suvorov
SPL_RSA_VERIFY requires SPL_RSA to be enabled. Add correct dependency. Signed-off-by: Oleksandr Suvorov <oleksandr.suvorov@foundries.io>
2021-09-16Merge tag 'v2021.10-rc4' into nextTom Rini
Prepare v2021.10-rc4 Signed-off-by: Tom Rini <trini@konsulko.com> # gpg: Signature made Tue 14 Sep 2021 06:58:32 PM EDT # gpg: using RSA key 1A3C7F70E08FAB1707809BBF147C39FF9634B72C # gpg: Good signature from "Thomas Rini <trini@konsulko.com>" [ultimate] # Conflicts: # board/Arcturus/ucp1020/spl.c # cmd/mvebu/Kconfig # common/Kconfig.boot # common/image-fit.c # configs/UCP1020_defconfig # configs/sifive_unmatched_defconfig # drivers/pci/Kconfig # include/configs/UCP1020.h # include/configs/sifive-unmatched.h # lib/Makefile # scripts/config_whitelist.txt
2021-09-14image: rsa: Move padding_algos to linker listsAlexandru Gagniuc
We are not guaranteed to have the padding_pkcs_15_verify symbol since commit 92c960bc1d ("lib: rsa: Remove #ifdefs from rsa.h"), and commit 61416fe9df ("Kconfig: FIT_SIGNATURE should not select RSA_VERIFY") The padding_algos only make sense with RSA verification, which can now be disabled in lieu of ECDSA. In fact this will lead to build failures because of the missing symbol mentioned earlier. To resolve this, move the padding_algos to a linker list, with declarations moved to rsa_verify.c. This is consistent with commit 6909edb4ce ("image: rsa: Move verification algorithm to a linker list") One could argue that the added #ifdef USE_HOSTCC is ugly, and should be hidden within the U_BOOT_PADDING_ALGO() macro. However, this would be inconsistent with the "cryptos" list. This logic for was not previously explored: Without knowledge of the U_BOOT_PADDING_ALGO() macro, its use is similar to something being declared. However, should #ifndef USE_HOSTCC be part of the macro, it would not be obvious that it behaves differently on host code and target code. Having the #ifndef outside the macro makes this obvious. Also, the #ifdef is not always necessary. For example ecda-verify makes use of U_BOOT_CRYPTO_ALGO() without any accompanying #ifdefs. The fundamental issue is a lack of separation of host and target code in rsa_verify. Therefore, the declaration of a padding algo with the external #ifdef is more readable and consistent. Signed-off-by: Alexandru Gagniuc <mr.nuke.me@gmail.com>
2021-09-14lib/rsa: don't use NULL as key_idHeinrich Schuchardt
If keydir is not provided but name is we want to use name as key_id. But with the current coding name is only used on its own if it is NULL and keydir is provided which never occurs. Fixes: 824ee745fbca ("lib/rsa: Use the 'keyfile' argument from mkimage") Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de> Reviewed-by: Simon Glass <sjg@chromium.org>
2021-09-01lib: -Wformat-truncation in rsa_engine_get_priv_keyHeinrich Schuchardt
With glibc 2.33 (Ubuntu package glibc6 2.33-0ubuntu9) building sifive_unmatched_defconfig results in: In file included from /usr/include/stdio.h:866, from ././include/compiler.h:26, from <command-line>: In function ‘snprintf’, inlined from ‘rsa_engine_get_priv_key’ at ./tools/../^:273:4: /usr/include/riscv64-linux-gnu/bits/stdio2.h:71:10: warning: ‘%s’ directive argument is null [-Wformat-truncation=] 71 | return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1, | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 72 | __glibc_objsize (__s), __fmt, | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 73 | __va_arg_pack ()); | ~~~~~~~~~~~~~~~~~ Avoid passing a NULL string. Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2021-09-01lib/rsa: Remove support for OpenSSL < 1.1.0 and libressl < 2.7.0Alexandru Gagniuc
Older OpenSSL and libressl versions have a slightly different API. This require #ifdefs to support. However, we still can't support it because the ECDSA path does not compile with these older versions. These #ifdefs are truly a vestigial appendage. Alternatively, the ECDSA path could be updated for older libraries, but this requires significant extra code, and #ifdefs. Those libraries are over three years old, and there concerns whether it makes sense to build modern software for real world use against such old libraries. Thusly, remove #ifdefs and code for old OpenSSL and LibreSSL support. Signed-off-by: Alexandru Gagniuc <mr.nuke.me@gmail.com>
2021-07-28mkimage: use environment variable MKIMAGE_SIGN_PIN to set pin for OpenSSL EngineMarc Kleine-Budde
This patch adds the possibility to pass the PIN the OpenSSL Engine used during signing via the environment variable MKIMAGE_SIGN_PIN. This follows the approach used during kernel module signing ("KBUILD_SIGN_PIN") or UBIFS image signing ("MKIMAGE_SIGN_PIN"). Signed-off-by: Marc Kleine-Budde <mkl@pengutronix.de>
2021-07-28lib: rsa: rsa-sign: Minor bug in debug messageChan, Donald
*sig_size isn't set until later so use the correct variables. Signed-off-by: Donald Chan <hoiho@lab126.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2021-07-28lib: rsa: rsa-verify: Fix a typo in a debug messageThomas Perrot
Signed-off-by: Thomas Perrot <thomas.perrot@bootlin.com>
2021-07-16image: rsa: Move verification algorithm to a linker listAlexandru Gagniuc
Move the RSA verification crytpo_algo structure out of the crypto_algos array, and into a linker list. Although it appears we are adding an #ifdef to rsa-verify.c, the gains outweigh this small inconvenience. This is because rsa_verify() is defined differently based on #ifdefs. This change allows us to have a single definition of rsa_verify(). Signed-off-by: Alexandru Gagniuc <mr.nuke.me@gmail.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2021-07-16image: Rename CONFIG_FIT_ENABLE_RSASSA_PSS_SUPPORTSimon Glass
Drop the ENABLE and SUPPORT parts of this, which are redundant. Signed-off-by: Simon Glass <sjg@chromium.org> Reviewed-by: Alexandru Gagniuc <mr.nuke.me@gmail.com> Signed-off-by: Alexandru Gagniuc <mr.nuke.me@gmail.com>
2021-04-14lib/rsa: Use EVP_PKEY instead of RSAChan, Donald
Most modern OpenSSL engines have methods overridden at the EVP level rather than at RSA level, to make these engines work properly with mkimage, the RSA signing code needs to switch to using EVP_* APIs as much as possible. Signed-off-by: Donald Chan <hoiho@lab126.com> [trini: Rebase on top of keyfile changes] Signed-off-by: Tom Rini <trini@konsulko.com>
2021-04-14lib/rsa: Use the 'keyfile' argument from mkimageAlexandru Gagniuc
Keys can be derived from keydir, and the "key-name-hint" property of the FIT. They can also be specified ad-literam via 'keyfile'. Update the RSA signing path to use the appropriate one. Signed-off-by: Alexandru Gagniuc <mr.nuke.me@gmail.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2021-04-14lib/rsa: Make fdt_add_bignum() available outside of RSA codeAlexandru Gagniuc
fdt_add_bignum() is useful for algorithms other than just RSA. To allow its use for ECDSA, move it to a common file under lib/. The new file is suffixed with '-libcrypto' because it has a direct dependency on openssl. This is due to the use of the "BIGNUM *" type. Signed-off-by: Alexandru Gagniuc <mr.nuke.me@gmail.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2021-04-14lib: Rename rsa-checksum.c to hash-checksum.cAlexandru Gagniuc
rsa-checksum.c sontains the hash_calculate() implementations. Despite the "rsa-" file prefix, this function is useful for other algorithms. To prevent confusion, move this file to lib/, and rename it to hash-checksum.c, to give it a more "generic" feel. Signed-off-by: Alexandru Gagniuc <mr.nuke.me@gmail.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2021-02-24lib: rsa: Add debug message on algo mismatchSean Anderson
Currently we fail silently if there is an algorithm mismatch. To help distinguish this failure condition. Signed-off-by: Sean Anderson <sean.anderson@seco.com>
2021-01-27lib: rsa: rsa-verify: don't look for keys in the FIT imagePhilippe Reynes
In the function rsa_verify_hash, if the "main" key doesn't work, u-boot try others keys. But it searches those keys in the FIT image instead of the u-boot device tree. Signed-off-by: Philippe Reynes <philippe.reynes@softathome.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2020-10-27xilinx: zynq: Change types from u32 to uint32_tMichal Simek
Change parameter type to avoid compilation error: In file included from ./tools/../lib/rsa/rsa-verify.c:23:0, from tools/lib/rsa/rsa-verify.c:1: include/u-boot/rsa-mod-exp.h:69:18: error: unknown type name ‘u32’; did you mean ‘__u32’? int zynq_pow_mod(u32 *keyptr, u32 *inout); ^~~ __u32 include/u-boot/rsa-mod-exp.h:69:31: error: unknown type name ‘u32’; did you mean ‘__u32’? int zynq_pow_mod(u32 *keyptr, u32 *inout); ^~~ __u32 Fixes: 37e3a36a5475 ("xilinx: zynq: Add support to secure images") Signed-off-by: Michal Simek <michal.simek@xilinx.com>
2020-10-12lib: rsa: superfluous initialization in rsa_verify()Heinrich Schuchardt
Remove initialization of ret with unused value. Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de> Reviewed-by: Simon Glass <sjg@chromium.org>
2020-10-12rsa: fix retrieving public exponent on big-endian systemsRasmus Villemoes
Commit fdf0819afb (rsa: fix alignment issue when getting public exponent) changed the logic to avoid doing an 8-byte access to a possibly-not-8-byte-aligned address. However, using rsa_convert_big_endian is wrong: That function converts an array of big-endian (32-bit) words with the most significant word first (aka a BE byte array) to an array of cpu-endian words with the least significant word first. While the exponent is indeed _stored_ as a big-endian 64-bit word (two BE words with MSW first), we want to extract it as a cpu-endian 64 bit word. On a little-endian host, swapping the words and byte-swapping each 32-bit word works, because that's the same as byte-swapping the whole 64 bit word. But on a big-endian host, the fdt32_to_cpu are no-ops, but rsa_convert_big_endian() still does the word-swapping, breaking verified boot. To fix that, while still ensuring we don't do unaligned accesses, add a little helper that first memcpy's the bytes to a local fdt64_t, then applies fdt64_to_cpu(). [The name is chosen based on the [bl]eXX_to_cpup in linux/byteorder/generic.h]. Fixes: fdf0819afb ("rsa: fix alignment issue when getting public exponent") Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk> Reviewed-by: Simon Glass <sjg@chromium.org>
2020-10-12lib: rsa: check algo match in rsa_verify_with_keynodeMatthieu CASTET
The algo name should match between the FIT's signature node and the U-Boot's control FDT. If we do not check it, U-Boot's control FDT can expect sha512 hash but nothing will prevent to accept image with sha1 hash if the signature is correct. Signed-off-by: Matthieu CASTET <castet.matthieu@free.fr>
2020-09-14rsa: crash in br_i32_decode() called from rsa_gen_key_prop()Robert Reither
Fixes problem for unaligned 32bit big-endian access in lib/rsa/rsa-keyprop.c. Exchanges br_i32_decode() with get_unaligned_be32(). This will keep the unaligned access for architectures capable and will do some byte-shift magic for the not so capable ones. Reported-by: Heinrich Schuchardt <xypron.glpk@gmx.de> Signed-by: Robert Reither <robert.reither@external.thalesgroup.com> Remove unused include. Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
2020-08-27lib/rsa: correct check after allocation in fdt_add_bignum()Heinrich Schuchardt
After allocating to pointer ctx we should check that pointer and not another pointer already checked above. Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de> Reviewed-by: Simon Glass <sjg@chromium.org>
2020-07-11lib: rsa: export rsa_verify_with_pkey()AKASHI Takahiro
This function will be used to implement public_key_verify_signature() in a later patch. rsa_verify() is not suitable here because calculation of message digest is not necessary. Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org> Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
2020-07-08lib: rsa: function to verify a signature against a hashHeiko Stuebner
rsa_verify() expects a memory region and wants to do the hashing itself, but there may be cases where the hashing is done via other means, like hashing a squashfs rootfs. So add rsa_verify_hash() to allow verifiying a signature against an existing hash. As this entails the same verification routines we can just move the relevant code over from rsa_verify() and also call rsa_verify_hash() from there. Signed-off-by: Heiko Stuebner <heiko.stuebner@theobroma-systems.com>
2020-07-08lib: rsa: add documentation to padding_pss_verify to document limitationsHeiko Stuebner
padding_pss_verify only works with the default pss salt setting of -2 (length to be automatically determined based on the PSS block structure) not -1 (salt length set to the maximum permissible value), which makes verifications of signatures with that saltlen fail. Until this gets implemented at least document this behaviour. Signed-off-by: Heiko Stuebner <heiko.stuebner@theobroma-systems.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2020-07-08lib: rsa: free local arrays after use in rsa_gen_key_prop()Heiko Stuebner
n, rr and rrtmp are used for internal calculations, but in the end the results are copied into separately allocated elements of the actual key_prop, so the n, rr and rrtmp elements are not used anymore when returning from the function and should of course be freed. Signed-off-by: Heiko Stuebner <heiko.stuebner@theobroma-systems.com> Reviewed-by: Simon Glass <sjg@chromium.org>
2020-07-08lib: rsa: fix allocated size for rr and rrtmp in rsa_gen_key_prop()Heiko Stuebner
When calculating rrtmp/rr rsa_gen_key_prop() tries to make (((rlen + 31) >> 5) + 1) steps in the rr uint32_t array and (((rlen + 7) >> 3) + 1) / 4 steps in uint32_t rrtmp[] with rlen being num_bits * 2 On a 4096bit key this comes down to to 257 uint32_t elements in rr and 256 elements in rrtmp but with the current allocation rr and rrtmp only have 129 uint32_t elements. On 2048bit keys this works by chance as the defined max_rsa_size=4096 allocates a suitable number of elements, but with an actual 4096bit key this results in other memory parts getting overwritten. So as suggested by Heinrich Schuchardt just use the actual bit-size of the key as base for the size calculation, in turn making the code compatible to any future keysizes. Suggested-by: Heinrich Schuchardt <xypron.debian@gmx.de> Signed-off-by: Heiko Stuebner <heiko.stuebner@theobroma-systems.com> Reviewed-by: Simon Glass <sjg@chromium.org> rrtmp needs 2 + (((*prop)->num_bits * 2) >> 5) array elements. Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
generated by cgit v1.2.3 (git 2.0.4) at 2024-06-27 03:45:28 +0000