Import performance improvements (ElementalX)#5
Closed
NewEraCracker wants to merge 10 commits into
Closed
Conversation
Changes: -------- v4->v3: - s/random_stack_user()/get_atrandom_bytes()/ - Move this function to ahead of its use to avoid the predeclaration. v3->v2: - Tweak code comments of random_stack_user(). - Remove redundant bits mask and shift upon the random variable. v2->v1: - Fix random copy to check up buffer length that are not 4-byte multiples. v3 can be found at: http://www.spinics.net/lists/linux-fsdevel/msg59597.html v2 can be found at: http://www.spinics.net/lists/linux-fsdevel/msg59418.html v1 can be found at: http://www.spinics.net/lists/linux-fsdevel/msg59128.html Thanks, -Jeff Entropy is quickly depleted under normal operations like ls(1), cat(1), etc... between 2.6.30 to current mainline, for instance: $ cat /proc/sys/kernel/random/entropy_avail 3428 $ cat /proc/sys/kernel/random/entropy_avail 2911 $cat /proc/sys/kernel/random/entropy_avail 2620 We observed this problem has been occurring since 2.6.30 with fs/binfmt_elf.c: create_elf_tables()->get_random_bytes(), introduced by f06295b ("ELF: implement AT_RANDOM for glibc PRNG seeding"). /* * Generate 16 random bytes for userspace PRNG seeding. */ get_random_bytes(k_rand_bytes, sizeof(k_rand_bytes)); The patch introduces a wrapper around get_random_int() which has lower overhead than calling get_random_bytes() directly. With this patch applied: $ cat /proc/sys/kernel/random/entropy_avail 2731 $ cat /proc/sys/kernel/random/entropy_avail 2802 $ cat /proc/sys/kernel/random/entropy_avail 2878 Analyzed by John Sobecki. Signed-off-by: Jie Liu <jeff.liu@oracle.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Al Viro <viro@zeniv.linux.org.uk> Cc: Andreas Dilger <aedilger@gmail.com> Cc: Alan Cox <alan@linux.intel.com> Cc: Arnd Bergmann <arnn@arndb.de> Cc: John Sobecki <john.sobecki@oracle.com> Cc: James Morris <james.l.morris@oracle.com> Cc: Jakub Jelinek <jakub@redhat.com> Cc: Ted Ts'o <tytso@mit.edu> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Kees Cook <keescook@chromium.org> Cc: Ulrich Drepper <drepper@redhat.com> Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
(cherry picked from commit 06af1c52c9ea234e0b1266cc0b52c3e0c6c8fe9f) The current lz4 compress buffer is 16kb on 32-bits, 32kb on 64-bits system. But, lz4 needs only 16kb on both. On 64-bits, this causes wasted cpu cycles for additional memset during every compression. In case of lz4hc, the current buffer size is (256kb + 8) on 32-bits, (512kb + 16) on 64-bits. But, lz4hc needs only (256kb + 2 * pointer) on both. This patch fixes these wrong compress buffer sizes for 64-bits. Signed-off-by: Bongkyu Kim <bongkyu.kim@lge.com> Cc: Chanho Min <chanho.min@lge.com> Cc: Yann Collet <yann.collet.73@gmail.com> Cc: Kyungsik Lee <kyungsik.lee@lge.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
Swap is disabled, so theres no reason to want to place memory into swap Signed-off-by: Joe Maples <joe@frap129.org> Signed-off-by: engstk <eng.stk@sapo.pt>
Measured latency within the kernel driver has been fluctuating between 2~7ms. Elevating the workqueues has made it more consistently in the 2~4ms range. alloc_ordered_workqueue() still makes the workqueue singlethreaded (internal __WQ_ORDERED flag), so this commit only adds WQ_HIGHPRI additionally. Signed-off-by: Park Ju Hyung <qkrwngud825@gmail.com> Signed-off-by: engstk <eng.stk@sapo.pt>
Running dm-crypt in a standard workqueue results in IO competing for CPU time with standard user apps, which can lead to pipeline bubbles and seriously degraded performance. Move to a WQ_HIGHPRI workqueue to protect against that. bug 25392275 Change-Id: I589149a31c7b5d322fe2ed5b2476b1f6e3d5ee6f Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
Running dm-verity in a standard workqueue results in IO competing for CPU time with standard user apps, which can lead to pipeline bubbles and seriously degraded performance. Move to a WQ_HIGHPRI workqueue to protect against that. bug 25392275 Change-Id: Ic65d7bd6f04e4d77780119e926a50e71323575f0 Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
Every current KDE system has process named ksysguardd polling files
below once in several seconds:
$ strace -e trace=open -p $(pidof ksysguardd)
Process 1812 attached
open("/etc/mtab", O_RDONLY|O_CLOEXEC) = 8
open("/etc/mtab", O_RDONLY|O_CLOEXEC) = 8
open("/proc/net/dev", O_RDONLY) = 8
open("/proc/net/wireless", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/proc/stat", O_RDONLY) = 8
open("/proc/vmstat", O_RDONLY) = 8
Hell knows what it is doing but speed up reading /proc/vmstat by 33%!
Benchmark is open+read+close 1.000.000 times.
BEFORE
$ perf stat -r 10 taskset -c 3 ./proc-vmstat
Performance counter stats for 'taskset -c 3 ./proc-vmstat' (10 runs):
13146.768464 task-clock (msec) # 0.960 CPUs utilized ( +- 0.60% )
15 context-switches # 0.001 K/sec ( +- 1.41% )
1 cpu-migrations # 0.000 K/sec ( +- 11.11% )
104 page-faults # 0.008 K/sec ( +- 0.57% )
45,489,799,349 cycles # 3.460 GHz ( +- 0.03% )
9,970,175,743 stalled-cycles-frontend # 21.92% frontend cycles idle ( +- 0.10% )
2,800,298,015 stalled-cycles-backend # 6.16% backend cycles idle ( +- 0.32% )
79,241,190,850 instructions # 1.74 insn per cycle
# 0.13 stalled cycles per insn ( +- 0.00% )
17,616,096,146 branches # 1339.956 M/sec ( +- 0.00% )
176,106,232 branch-misses # 1.00% of all branches ( +- 0.18% )
13.691078109 seconds time elapsed ( +- 0.03% )
^^^^^^^^^^^^
AFTER
$ perf stat -r 10 taskset -c 3 ./proc-vmstat
Performance counter stats for 'taskset -c 3 ./proc-vmstat' (10 runs):
8688.353749 task-clock (msec) # 0.950 CPUs utilized ( +- 1.25% )
10 context-switches # 0.001 K/sec ( +- 2.13% )
1 cpu-migrations # 0.000 K/sec
104 page-faults # 0.012 K/sec ( +- 0.56% )
30,384,010,730 cycles # 3.497 GHz ( +- 0.07% )
12,296,259,407 stalled-cycles-frontend # 40.47% frontend cycles idle ( +- 0.13% )
3,370,668,651 stalled-cycles-backend # 11.09% backend cycles idle ( +- 0.69% )
28,969,052,879 instructions # 0.95 insn per cycle
# 0.42 stalled cycles per insn ( +- 0.01% )
6,308,245,891 branches # 726.058 M/sec ( +- 0.00% )
214,685,502 branch-misses # 3.40% of all branches ( +- 0.26% )
9.146081052 seconds time elapsed ( +- 0.07% )
^^^^^^^^^^^
vsnprintf() is slow because:
1. format_decode() is busy looking for format specifier: 2 branches
per character (not in this case, but in others)
2. approximately million branches while parsing format mini language
and everywhere
3. just look at what string() does /proc/vmstat is good case because
most of its content are strings
Link: http://lkml.kernel.org/r/20160806125455.GA1187@p183.telecom.by
Signed-off-by: Alexey Dobriyan <adobriyan@gmail.com>
Cc: Joe Perches <joe@perches.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
flar2
pushed a commit
that referenced
this pull request
Dec 8, 2016
Current mechanism to count number of synchronized ping pong transfers
can have a race condition in case ping pong done happens as in
following simplified code sequence:
Thread 0 | Thread 1
-------------------------|----------------------------
1) | wait_event(w, !ctx->koff_cnt &&
| !sctx->koff_cnt)
| /* ctx->koff_cnt == 1 */
| /* sctx->koff_cnt == 1 */
2) ctx->koff_cnt-- |
| /* ctx->koff_cnt == 0 */
3) if (!sctx->koff_cnt) |
4) pp_done_cnt++ |
5) sctx->koff_cnt-- |
| /* sctx->koff_cnt == 0 */
| /* wait_event unblocks, transfer done */
6) | ctx->koff_cnt++ /* new transfer */
7) | sctx->koff_cnt++
8) if (!ctx->koff_cnt) |
9) pp_done_cnt++ |
By decrementing koff_cnt at #5, we are signaling that transfer has
completed for both ctx and sctx, unblocking Thread 1 which can go
ahead and trigger next transfer before we increment pp_done_cnt. By
checking the status of ctx->koff_cnt before #5, we can safely allow
Thread 1 to go forward but still maintain proper count of pp_done_cnt.
CRs-Fixed: 966529
Change-Id: I59e7798b5cc5bc28ff56551a0a009503019d32a9
Signed-off-by: Adrian Salido-Moreno <adrianm@codeaurora.org>
In the arm64 arch_static_branch implementation we place an A64 NOP into the instruction stream and log relevant details to a jump_entry in a __jump_table section. Later this may be replaced with an immediate branch without link to the code for the unlikely case. At init time, the core calls arch_jump_label_transform_static to initialise the NOPs. On x86 this involves inserting the optimal NOP for a given microarchitecture, but on arm64 we only use the architectural NOP, and hence replace each NOP with the exact same NOP. This is somewhat pointless. Additionally, at module load time we don't call jump_label_apply_nops to patch the optimal NOPs in, unlike other architectures, but get away with this because we only use the architectural NOP anyway. A later notifier will patch NOPs with branches as required. Similarly to x86 commit 11570da (x86/jump-label: Do not bother updating NOPs if they are correct), we can avoid patching NOPs with identical NOPs. Given that we only use a single NOP encoding, this means we can NOP-out the body of arch_jump_label_transform_static entirely. As the default __weak arch_jump_label_transform_static implementation performs a patch, we must use an empty function to achieve this. Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Jiang Liu <liuj97@gmail.com> Cc: Laura Abbott <lauraa@codeaurora.org> Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com> (cherry picked from commit 6ddae41) Signed-off-by: Tomasz Figa <tfiga@chromium.org> Bug: 24475017 Change-Id: I9ef07c3a5a7f91418b30006850fcc0c421e147e2 Signed-off-by: Kees Cook <keescook@google.com> Signed-off-by: Francisco Franco <franciscofranco.1990@gmail.com>
Optimize memcpy and memmove, to prefetch several cache lines. We can achieve 15% memcpy speed improvement with the preload method. Change-Id: I2259b98a33eba0b7466920b3f270f953e609cf13 Signed-off-by: Hong-Mei Li <a21834@motorola.com> Reviewed-on: http://gerrit.mot.com/740766 SLTApproved: Slta Waiver <sltawvr@motorola.com> SME-Granted: SME Approvals Granted Tested-by: Jira Key <jirakey@motorola.com> Reviewed-by: Zhi-Ming Yuan <a14194@motorola.com> Submit-Approved: Jira Key <jirakey@motorola.com> Signed-off-by: engstk <eng.stk@sapo.pt>
We want to avoid lots of different copy_page implementations, settling
for something that is "good enough" everywhere and hopefully easy to
understand and maintain whilst we're at it.
This patch reworks our copy_page implementation based on discussions
with Cavium on the list and benchmarking on Cortex-A processors so that:
- The loop is unrolled to copy 128 bytes per iteration
- The reads are offset so that we read from the next 128-byte block
in the same iteration that we store the previous block
- Explicit prefetch instructions are removed for now, since they hurt
performance on CPUs with hardware prefetching
- The loop exit condition is calculated at the start of the loop
Change-Id: I0d9f3bbe4efa2751f41432a3b4b299fbb0e494be
Signed-off-by: Will Deacon <will.deacon@arm.com>
Tested-by: Andrew Pinski <apinski@cavium.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: engstk <eng.stk@sapo.pt>
NewEraCracker
changed the title
flar2
pushed a commit
that referenced
this pull request
May 24, 2017
When I forced to enable atomic operations intentionally, I could hit the below panic, since we didn't clear page->private in f2fs_invalidate_page called by file truncation. The panic occurs due to NULL mapping having page->private. BUG: unable to handle kernel paging request at ffffffffffffffff IP: drop_buffers+0x38/0xe0 PGD 5d00c067 PUD 5d00e067 PMD 0 CPU: 3 PID: 1648 Comm: fsstress Tainted: G D OE 4.10.0+ #5 Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006 task: ffff9151952863c0 task.stack: ffffaaec40db4000 RIP: 0010:drop_buffers+0x38/0xe0 RSP: 0018:ffffaaec40db74c8 EFLAGS: 00010292 Call Trace: ? page_referenced+0x8b/0x170 try_to_free_buffers+0xc5/0xe0 try_to_release_page+0x49/0x50 shrink_page_list+0x8bc/0x9f0 shrink_inactive_list+0x1dd/0x500 ? shrink_active_list+0x2c0/0x430 shrink_node_memcg+0x5eb/0x7c0 shrink_node+0xe1/0x320 do_try_to_free_pages+0xef/0x2e0 try_to_free_pages+0xe9/0x190 __alloc_pages_slowpath+0x390/0xe70 __alloc_pages_nodemask+0x291/0x2b0 alloc_pages_current+0x95/0x140 __page_cache_alloc+0xc4/0xe0 pagecache_get_page+0xab/0x2a0 grab_cache_page_write_begin+0x20/0x40 get_read_data_page+0x2e6/0x4c0 [f2fs] ? f2fs_mark_inode_dirty_sync+0x16/0x30 [f2fs] ? truncate_data_blocks_range+0x238/0x2b0 [f2fs] get_lock_data_page+0x30/0x190 [f2fs] __exchange_data_block+0xaaf/0xf40 [f2fs] f2fs_fallocate+0x418/0xd00 [f2fs] vfs_fallocate+0x157/0x220 SyS_fallocate+0x48/0x80 Signed-off-by: Yunlei He <heyunlei@huawei.com> Signed-off-by: Chao Yu <yuchao0@huawei.com> [Chao Yu: use INMEM_INVALIDATE for better tracing] Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org> Conflicts: include/trace/events/f2fs.h
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Improves performance using commits from eng.stk and Franco kernels. Upstream: OnePlusOSS#23
Thanks!
Regards,
Jorge M. Oliveira
Edit: All fixes tested and no regressions found :)