Memory and Scan Rate

Programming and Web Development Forums - unix - Discussions about the Solaris operating system

Skip to content

Memory and Scan Rate

unix
Post a reply

    Next

  • 1. system accounting and space on /var
    Solaris 7 SPARC. I need to enable system accounting and have done so on many systems. My problem is that /var isn't big enough on the systems that are really active. Too much data being gathered fills up /var with pacctX files. I would rather not rebuild the system just to fix that. I was wondering if I can get around this by changing user adm's homedir in the /etc/passwd file to /someother/path as opposed to /var/adm. If I did, would I break anything else ? The /someother/path of course is where there's lots of disk space for accounting data. TIA
  • 2. [help] problems running prstat from crontab
    Hi everyone, I encoutered problems scheduling prstat in the crontab as follow: If i put "prstat -n 15 5 10 >prtstat.out" (ten times, every 5 seconds and display the first 15 lines) in the crontab, when it starts doesn't wait for 5 seconds to puts the next report in prstat.out, but puts all the ten reports at the same moment. if i run the same command from shell , it's OK. If i run iostat instead of prstat it's OK, either from crontab either from shell. Caius from Rome P.S. I hope u understand my bad English :-)
  • 3. race on multi-processor solaris
    I ran into a race condition when I tried to run my multi threads program on solaris. My main thread had the lowest priority than other threads I created and should be the last one to get running. But it looked like the main thread got running on a different processor and created a race condition. I am wondering if there are some function calls that I can use to assign all the threads to a specific processor or use some locking mechanism to fix this race? Thanks for your help! Hong Chen
  • 4. Systemwide environment variables?
    Hi I use /etc/csh.login to set environment variables for all users. But it does not work. So I tried /etc/csh.cshrc, but not work too. I have to source every time I turn on the machine. I want the machine do it automatically. Need yours HELP. Ngoc
  • 5. Pine 4.58 with SSL
    Anyone out there have Pine 4.58 with SSL built already? Can I snag it? -Jason XXXX@XXXXX.COM

Memory and Scan Rate

Postby noident » Tue, 05 Dec 2006 14:53:30 GMT 

Greetings!
We are running Solaris 10 3/05.
The system has run out of memory a few times due to different causes,
such as memory leaks and dodgy scripts written by some developers.
What's interesting is that vmstat's scan rate (the sr column) shows 0.
Shouldn't the page scanner have kicked in and Scan Rate be a non-zero
value?

Cheers.
Top

Re: Memory and Scan Rate

Postby Darren Dunham » Tue, 05 Dec 2006 17:11:38 GMT 



It depends on exactly how you ran out of memory.

The page scanner tries to find free RAM.  If you reserve but don't
immediately use a lot of memory (think alloc() ), then you'll run out of
virtual memory.  The page scanner can't do anything about that.

So if you just alloc(), all over the place, you'll run out of VM and I
would expect the page scanner to never get involved.

Do you have any more detail about exactly what happened in the past?

-- 
Darren Dunham                                            XXXX@XXXXX.COM 
Senior Technical Consultant         TAOS             http://www.**--****.com/ 
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >
Top

Re: Memory and Scan Rate

Postby noident » Wed, 06 Dec 2006 04:58:24 GMT 

> Do you have any more detail about exactly what happened in the past?
Well, the first time we ran out of memory was when an sql query
triggered what I think was a memory leak - where mysqld's size and
rsize were increasing for about an hour until nothing on the system
could allocate any more memory. The process was increasing its memory
size at about 2 MB per second. With 16GB RAM, I would expect the page
scanner to kick in once the free RAM falls below 250MB, which I believe
it had plenty of time to do. But it didn't somehow...

The 2nd time around it was a dodgy perl script that put too much stuff
in hashes and ran out of memory in about 5 minutes.
Top

Re: Memory and Scan Rate

Postby Darren Dunham » Wed, 06 Dec 2006 08:11:54 GMT 



At the end of it, what kind of error did you have?  How much free swap
(esitmated) did you have?  (No free swap, nothing for page scanner to
do).

If you run out of VM, you'll get a "mem alloc failed" or something like
that.  If you instead run out of RAM, nothing should happen, but it
might be slow until the page scanner finds some space.  The process
shouldn't die (or notice).


If I remember, I'll try to write a few one-liners to kill a machine
these ways.  You can then watch the numbers as they drop and either see
it die or the page scanner kick in.  I don't have a non-production
machine to run it on at the moment.

The idea is to just malloc() all over the place (should run out of VM,
but not RAM), and then again with calloc() instead (should dry up RAM
first).

-- 
Darren Dunham                                            XXXX@XXXXX.COM 
Senior Technical Consultant         TAOS             http://www.**--****.com/ 
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >
Top

Re: Memory and Scan Rate

Postby noident » Wed, 06 Dec 2006 08:30:47 GMT 

> At the end of it, what kind of error did you have?  How much free swap

Sorry, I had to be more specific - ran out of VM completely.
The error messages were "Out of memory", "Couldn't allocate memory",
etc.
That is malloc() and similar returned NULL.
Top

Re: Memory and Scan Rate

Postby Darren Dunham » Wed, 06 Dec 2006 10:38:28 GMT 




So, I'm not sure why you didn't see page scanning beforehand, but
running out of VM isn't an issue that involves the page scanner at all.
It can't create any more VM.

If you get to that point, then things must be working (in some sense).
The system has successfully allocated all VM pages.  

-- 
Darren Dunham                                            XXXX@XXXXX.COM 
Senior Technical Consultant         TAOS             http://www.**--****.com/ 
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >
Top

Re: Memory and Scan Rate

Postby Darren Dunham » Wed, 06 Dec 2006 15:27:44 GMT

arren Dunham < XXXX@XXXXX.COM > wrote:

Okay. Here is a run with such. Program is at the end so you can try it
out.

The following is a run where I'm allocating ( malloc() ) 10 megabytes a
second, and I'm running vmstat with a 5 second interval. The machine
has 128Mb RAM and a 2GB swap partition. Let's skip to the end.

[...]
0 0 0 376504 15792 0 2 0 0 0 0 0 0 0 0 0 404 24 44 0 1 99
0 0 0 325248 15656 0 2 0 0 0 0 0 0 0 0 0 404 27 46 0 1 99
0 0 0 276048 15536 0 2 0 0 0 0 0 0 0 0 0 402 32 50 0 1 99
0 0 0 233000 15424 0 2 0 0 0 0 0 0 0 0 0 405 23 42 0 1 99
0 0 0 181752 15296 0 2 0 0 0 0 0 0 0 0 0 404 26 45 0 1 99
0 0 0 130504 15168 0 2 0 0 0 0 0 0 0 0 0 403 26 43 0 1 99
0 0 0 79248 15032 0 2 0 0 0 0 0 0 0 0 0 405 28 52 0 1 99
0 0 0 28000 14904 0 2 0 0 0 0 0 0 0 0 0 404 36 47 0 1 99
0 0 0 2106832 18248 0 0 0 0 0 0 0 0 0 0 0 401 15 37 0 1 99

It exits immediately after a failed alloc and returns the memory. The
free RAM is bouncing around a little bit, but not 10MB/sec.

Next, I use calloc() instead of malloc().

# vmstat 5
kthr memory page disk faults cpu
r b w swap free re mf pi po fr de sr dd s1 -- -- in sy cs us sy id
0 0 0 2047824 15040 4 21 21 0 0 0 6 3 0 0 0 408 284 182 1 2 97
0 0 0 2096040 9552 1 451 0 0 48 0 771 0 0 0 0 405 47 62 2 6 92
0 0 0 2074376 2440 0 806 9 4433 7041 0 3013 50 0 0 0 503 24 187 3 14 83
0 0 0 2037720 2048 0 699 66 4942 5416 0 3975 49 0 0 0 501 18 246 3 13 84
0 0 0 2006608 1872 1 664 60 5158 5172 0 4145 50 0 0 0 502 17 263 3 13 85
0 0 0 1975608 1704 1 618 168 5196 5225 0 4288 61 0 0 0 526 18 278 2 13 85
[...]
0 0 0 140392 1992 0 653 110 4966 4982 0 3874 60 0 0 0 511 20 297 3 13 85
0 0 0 110592 1304 1 662 268 5346 5401 0 4797 178 0 0 0 763 18 553 3 16 81
0 0 0 77448 1816 0 670 108 5424 5508 0 4632 99 0 0 0 601 19 375 3 14 83
0 0 0 47560 2128 0 634 114 4843 4849 0 3651 78 0 0 0 560 20 345 2 13 85
0 0 0 15424 5088 2 283 124 3491 3548 0 3238 216 0 0 0 835 10 383 1 12 87
5 0 0 9384 14672 237 15 84 0 0 0 0 8 0 0 0 418 14 35 0 88 12
0 0 0 2087376 63280 1 48 492 0 0 0 0 48 0 0 0 500 36 135 0 2 98


Wham.. Because this is a little, underpowered Ultra 5, it runs out of
free RAM almost immediately and has to begin scanning to gather free
pages. Also notice the machine had a lot more work to do when it came
time to actually free all those pages.

I'm not sure I understand the numbers, though. Since I'm allocating
10240 KB/s, I expected to see a similar figure in 'po', but it's only
about half that; not sure why.

Here's the program.

/* overalloc.c
Attempt to allocate memory until error occurs
Then print allocation amount and exit. */

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main()
{
size_t alloc_size = 1024 * 1024 * 10; /* 10 MB */
size_t nelem = 1;
int alloc_num = 0;
void *buf_ptr;

while (1)
{
alloc_num++;
/* buf_ptr = malloc(alloc_size); */ /* Change comments for */
buf_ptr = calloc(nelem, alloc_size); /* calloc vs alloc */

if (buf_ptr == NULL)
{
printf ("Alloc failed
Top

Similar Threads:

1.VMM scan rate and free list 

Hi,
I am confused about the following.
I have a 5.2 lpar which when running vmstat shows that the server is
scanning at well more than 10:1. However the free list shows that I
constantly have approx 1Gb memory available.
The server is not paging either.
What is causing the scan rate to be so high?

System Configuration: lcpu=8 mem=12288MB
kthr     memory             page              faults        cpu
----- ----------- ------------------------ ------------ -----------
 r  b   avm   fre  re  pi  po  fr   sr  cy  in   sy  cs us sy id wa
 5  7 1288610 278954   0   0   0 2522  410   0 2251 43451 23570 19  8
63 11
 2  2 1289288 278253   0   0   0 222 5128   0 1727 52101 17523 13  8 70
 9
 1  1 1298869 269071   0   0   0 201 3285   0 1527 73714 16595 13  9 72
 6
 3  1 1295729 272176   0   0   0 340 5207   0 1519 59766 13243 13  8 71
 8
 2  1 1295397 272521   0   0   0 155 2152   0 1339 24606 14234 25  4 63
 7
 5  2 1299715 268189   0   0   0 573 7591   0 1811 57357 7944 25  9 45
21

2.How can I compute the scan rate (sr) ???

3.[PATCH 06/12] mm: balance active/inactive list scan rates

4.[PATCH 00/12] Balancing the scan rate of major caches

5.Scanning processes memory for a string 

Hi there
I am apparently struck by a *very* smart rootkit, probably a LKM. Some 
unknown process or kernel thread on our system is making a spam-network 
with some other peers. Netstat won't show that process (not even with a 
dash "-" like it is for kernel threads like NFS). "ps" probably also 
won't show it.

The peer IP addresses which are contacted by the infected machine are 
recurring over time however, so there must be some running process that 
holds those IP addresses in memory.

Is there a way I can use /dev/mem or /dev/kmem to scan for those IP 
addresses (double word value)? After I found it, I would need to know 
which process maps that portion of memory. (That might even be a kernel 
thread actually) Can you point me to some direction? Consider that I 
know C/C++ but I don't really know much of the linux kernel.

Thank you

6. Memory scanning under Linux

7. [PATCH] fix memory leak in UBI scanning unit

8. [PATCH 1/2] limit print_fatal_signal() rate (was: [RFC] log out-of-virtual-memory events)

Post a reply

Return to unix

 

Who is online

Users browsing this forum: No registered users and 24 guest

Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group