This post is an updated version of the original post , with the code examples below reflecting changes in response to very detailed inputs from user @pacmaninbw. The updated source examples below also include the addition of a function that is designed to accommodate commands that do not return a response. The comment block describes why this function addition is necessary, and its usage.
I am interested in getting feedback with emphasis on the same things as before, so will repeat the preface of the previous post below.
The need:
I needed a method to programmatically send commands to the Windows 7 CMD prompt, and return the response without seeing a console popup in multiple applications.
The design:
The environment in addition to the Windows 7 OS is an ANSI C (C99) compiler from National Instruments, and the Microsoft Windows Driver Kit for Windows 8.1. Among the design goals was to present a very small API, including well documented and straightforward usage instructions. The result is two exported functions. Descriptions for each are provided in their respective comment blocks. In its provided form, it is intended to be built as a DLL. The only header files used in this library are
windows.h
andstdlib.h
.For review consideration:
The code posted is complete, and I have tested it, but I am new to using
pipes
tostdin
andstdout
, as well as using Windows methods forCreateProcess(...)
. Also, because the size requirements of the response buffer cannot be known at compile time, the code includes the ability to grow the response buffer as needed during run-time. For example, I have used this code to recursively read directories usingdir /s
from all locations except thec:\
directory with the following command:cd c:\dev && dir /s // approximately 1.8Mbyte buffer is returned on my system
I would especially appreciate feedback focused on the following:
- Pipe creation and usage
CreateProcess
usage- Method for dynamically growing response buffer (very interested in feedback on this)
- Handling embedded
null
bytes in content returned fromReadFile
function. (Credit to @chux, this is a newly discovered deficiency)
Usage example:
#include <stdio.h> // printf()
#include <stdlib.h> // NULL
#include "cmd_rsp.h"
#define BUF_SIZE 100
int main(void)
{
char *buf = NULL;
/// test cmd_rsp
buf = calloc(BUF_SIZE, 1);
if(!buf)return 0;
if (!cmd_rsp("dir /s", &buf, BUF_SIZE))
{
printf("%s", buf);
}
else
{
printf("failed to send command.\n");
}
free(buf);
/// test cmd_no_rsp
buf = calloc(BUF_SIZE, 1);
if(!buf)return 0;
if (!cmd_no_rsp("dir /s", &buf, BUF_SIZE))
{
printf("success.\n"); // function provides no response
}
else
{
printf("failed to send command.\n");
}
free(buf);
return 0;
}
cmd_rsp.h
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//
// Prototype: int int __declspec(dllexport) cmd_rsp(char *command, char **chunk, size_t size)
//
// Description: Executes any command that can be executed in a Windows cmd prompt and returns
// the response via auto-resizing buffer.
/// Note: this function will hang for executables or processes that run and exit
/// without ever writing to stdout.
/// The hang occurs during the call to the read() function.
//
// Inputs: const char *command - string containing complete command to be sent
// char **chunk - initialized pointer to char array to return results
// size_t size - Initial memory size in bytes char **chunk was initialized to.
//
// Return: 0 for success
// -1 for failure
//
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////
int __declspec(dllexport) cmd_rsp(const char *command, char **chunk, unsigned int chunk_size);
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//
// Prototype: int int __declspec(dllexport) cmd_no_rsp(char *command)
//
// Description: Variation of cmd_rsp that does not wait for a response. This is useful for
// executables or processes that run and exit without ever sending a response to stdout,
// causing cmd_rsp to hang during the call to the read() function.
//
// Inputs: const char *command - string containing complete command to be sent
//
// Return: 0 for success
// -1 for failure
//
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////
int __declspec(dllexport) cmd_no_rsp(const char *command);
#endif
cmd_rsp.c
#include <windows.h>
#include <stdlib.h> // calloc, realloc & free
#include "cmd_rsp.h"
#define BUFSIZE 1000
typedef struct {
/* child process's STDIN is the user input or data entered into the child process - READ */
void * in_pipe_read;
/* child process's STDIN is the user input or data entered into the child process - WRITE */
void * in_pipe_write;
/* child process's STDOUT is the program output or data that child process returns - READ */
void * out_pipe_read;
/* child process's STDOUT is the program output or data that child process returns - WRITE */
void * out_pipe_write;
}IO_PIPES;
// Private prototypes
static int CreateChildProcess(const char *cmd, IO_PIPES *io);
static int CreateChildProcessNoStdOut(const char *cmd, IO_PIPES *io);
static int ReadFromPipe(char **rsp, unsigned int size, IO_PIPES *io);
static char * ReSizeBuffer(char **str, unsigned int size);
static void SetupSecurityAttributes(SECURITY_ATTRIBUTES *saAttr);
static void SetupStartUpInfo(STARTUPINFO *siStartInfo, IO_PIPES *io);
static int SetupChildIoPipes(IO_PIPES *io, SECURITY_ATTRIBUTES *saAttr);
int __stdcall DllMain (HINSTANCE hinstDLL, DWORD fdwReason, LPVOID lpvReserved)
{
switch (fdwReason)
{
case DLL_PROCESS_ATTACH:
/* Respond to DLL loading by initializing the RTE */
if (InitCVIRTE (hinstDLL, 0, 0) == 0) return 0;
break;
case DLL_PROCESS_DETACH:
/* Respond to DLL unloading by closing the RTE for its use */
if (!CVIRTEHasBeenDetached ()) CloseCVIRTE ();
break;
}
/* Return 1 to indicate successful initialization */
return 1;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//
// Prototype: int __declspec(dllexport) cmd_rsp(char *command, char **chunk, size_t chunk_size)
//
// Description: Executes any command that can be executed in a Windows cmd prompt and returns
// the response via auto-resizing buffer.
//
// Inputs: const char *command - string containing complete command to be sent
// char **chunk - initialized pointer to char array to return results
// size_t chunk_size - Initial memory size in bytes of char **chunk.
//
// Return: 0 for success
// -1 for failure
//
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////
int __declspec(dllexport) cmd_rsp(const char *command, char **chunk, unsigned int chunk_size)
{
SECURITY_ATTRIBUTES saAttr;
/// All commands that enter here must contain (and start with) the substring: "cmd.exe /c
/// /////////////////////////////////////////////////////////////////////////////////////////////////////////
/// char cmd[] = ("cmd.exe /c \"dir /s\""); /// KEEP this comment until format used for things like
/// directory command (i.e. two parts of syntax) is captured
/// /////////////////////////////////////////////////////////////////////////////////////////////////////////
const char rqdStr[] = {"cmd.exe /c "};
int len = (int)strlen(command);
char *Command = NULL;
int status = 0;
Command = calloc(len + sizeof(rqdStr), 1);
if(!Command) return -1;
strcat(Command, rqdStr);
strcat(Command, command);
SetupSecurityAttributes(&saAttr);
IO_PIPES io;
if(SetupChildIoPipes(&io, &saAttr) < 0) return -1;
//eg: CreateChildProcess("adb");
if(CreateChildProcess(Command, &io) == 0)
{
// Read from pipe that is the standard output for child process.
ReadFromPipe(chunk, chunk_size, &io);
status = 0;
}
else
{
status = -1;
}
free(Command);
return status;
}
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//
// Prototype: int __declspec(dllexport) cmd_no_rsp(char *command)
//
// Description: Variation of cmd_rsp that does not wait for a response. This is useful for
// executables or processes that run and exit without ever sending a response to stdout,
// causing cmd_rsp to hang during the call to the read() function.
//
// Inputs: const char *command - string containing complete command to be sent
//
// Return: 0 for success
// -1 for failure
//
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////
int __declspec(dllexport) cmd_no_rsp(const char *command)
{
/// All commands that enter here must contain (and start with) the substring: "cmd.exe /c
/// /////////////////////////////////////////////////////////////////////////////////////////////////////////
/// char cmd[] = ("cmd.exe /c \"dir /s\""); /// KEEP this comment until format used for things like
/// directory command (i.e. two parts of syntax) is captured
/// /////////////////////////////////////////////////////////////////////////////////////////////////////////
SECURITY_ATTRIBUTES saAttr;
const char rqdStr[] = {"cmd.exe /c "};
int len = (int)strlen(command);
char *Command = NULL;
int status = 0;
Command = calloc(len + sizeof(rqdStr), 1);
if(!Command) return -1;
strcat(Command, rqdStr);
strcat(Command, command);
SetupSecurityAttributes(&saAttr);
IO_PIPES io;
if(SetupChildIoPipes(&io, &saAttr) < 0) return -1;
status = CreateChildProcessNoStdOut(Command, &io);
free(Command);
return status;
}
static int SetupChildIoPipes(IO_PIPES *io, SECURITY_ATTRIBUTES *saAttr)
{
//child process's STDOUT is the program output or data that child process returns
// Create a pipe for the child process's STDOUT.
if (!CreatePipe(&io->out_pipe_read, &io->out_pipe_write, saAttr, 0))
{
return -1;
}
// Ensure the read handle to the pipe for STDOUT is not inherited.
if (!SetHandleInformation(io->out_pipe_read, HANDLE_FLAG_INHERIT, 0))
{
return -1;
}
//child process's STDIN is the user input or data entered into the child process
// Create a pipe for the child process's STDIN.
if (!CreatePipe(&io->in_pipe_read, &io->in_pipe_write, saAttr, 0))
{
return -1;
}
// Ensure the write handle to the pipe for STDIN is not inherited.
if (!SetHandleInformation(io->in_pipe_write, HANDLE_FLAG_INHERIT, 0))
{
return -1;
}
return 0;
}
// Create a child process that uses the previously created pipes for STDIN and STDOUT.
static int CreateChildProcess(const char *cmd, IO_PIPES *io)
{
PROCESS_INFORMATION piProcInfo;
STARTUPINFO siStartInfo;
BOOL bSuccess = FALSE;
// Set up members of the PROCESS_INFORMATION structure.
ZeroMemory(&piProcInfo, sizeof(PROCESS_INFORMATION));
// Set up members of the STARTUPINFO structure.
// This structure specifies the STDIN and STDOUT handles for redirection.
ZeroMemory(&siStartInfo, sizeof(STARTUPINFO));
SetupStartUpInfo(&siStartInfo, io);
// Create the child process.
bSuccess = CreateProcess(NULL,
cmd, // command line
NULL, // process security attributes
NULL, // primary thread security attributes
TRUE, // handles are inherited
CREATE_NO_WINDOW, // creation flags
//CREATE_NEW_CONSOLE, // creation flags
NULL, // use parent's environment
NULL, // use parent's current directory
&siStartInfo, // STARTUPINFO pointer
&piProcInfo); // receives PROCESS_INFORMATION
// If an error occurs, exit the application.
if (!bSuccess)
{
return -1;
}
else
{
// Close handles to the child process and its primary thread.
CloseHandle(piProcInfo.hProcess);
CloseHandle(piProcInfo.hThread);
CloseHandle(io->out_pipe_write);
}
return 0;
}
// Create a child process that uses the previously created pipes for STDIN and STDOUT.
static int CreateChildProcessNoStdOut(const char *cmd, IO_PIPES *io)
{
PROCESS_INFORMATION piProcInfo;
STARTUPINFO siStartInfo;
BOOL bSuccess = FALSE;
// Set up members of the PROCESS_INFORMATION structure.
ZeroMemory(&piProcInfo, sizeof(PROCESS_INFORMATION));
// Set up members of the STARTUPINFO structure.
// This structure specifies the STDIN and STDOUT handles for redirection.
ZeroMemory(&siStartInfo, sizeof(STARTUPINFO));
SetupStartUpInfo(&siStartInfo, io);
// Create the child process.
bSuccess = CreateProcess(NULL,
cmd, // command line
NULL, // process security attributes
NULL, // primary thread security attributes
TRUE, // handles are inherited
CREATE_NO_WINDOW, // creation flags
//CREATE_NEW_CONSOLE, // creation flags
NULL, // use parent's environment
NULL, // use parent's current directory
&siStartInfo, // STARTUPINFO pointer
&piProcInfo); // receives PROCESS_INFORMATION
// If an error occurs, exit the application.
if (!bSuccess)
{
return -1;
}
else
{
// Close handles to the child process and its primary thread.
CloseHandle(piProcInfo.hProcess);
CloseHandle(piProcInfo.hThread);
CloseHandle(io->out_pipe_write);
}
return 0;
}
// Read output from the child process's pipe for STDOUT
// Grow the buffer as needed
// Stop when there is no more data.
static int ReadFromPipe(char **rsp, unsigned int size, IO_PIPES *io)
{
COMMTIMEOUTS ct;
int size_recv = 0;
unsigned int total_size = 0;
unsigned long dwRead;
BOOL bSuccess = TRUE;
char *accum;
char *tmp1 = NULL;
char *tmp2 = NULL;
//Set timeouts for stream
ct.ReadIntervalTimeout = 0;
ct.ReadTotalTimeoutMultiplier = 0;
ct.ReadTotalTimeoutConstant = 10;
ct.WriteTotalTimeoutConstant = 0;
ct.WriteTotalTimeoutMultiplier = 0;
SetCommTimeouts(io->out_pipe_read, &ct);
//This accumulates each read into one buffer,
//and copies back into rsp before leaving
accum = (char *)calloc(1, sizeof(char)); //grow buf as needed
if(!accum) return -1;
memset(*rsp, 0, size);
do
{
//Reads stream from child stdout
bSuccess = ReadFile(io->out_pipe_read, *rsp, size-1, &dwRead, NULL);
if (!bSuccess || dwRead == 0)
{
free(accum);
return 0;//successful - reading is done
}
(*rsp)[dwRead] = 0;
size_recv = (int)strlen(*rsp);
if(size_recv == 0)
{
//should not get here for streaming
(*rsp)[total_size]=0;
return total_size;
}
else
{
//New Chunk:
(*rsp)[size_recv]=0;
//capture increased byte count
total_size += size_recv+1;
//increase size of accumulator
tmp1 = ReSizeBuffer(&accum, total_size);
if(!tmp1)
{
free(accum);
strcpy(*rsp, "");
return -1;
}
accum = tmp1;
strcat(accum, *rsp);
if(total_size > (size - 1))
{ //need to grow buffer
tmp2 = ReSizeBuffer(&(*rsp), total_size+1);
if(!tmp2)
{
free(*rsp);
return -1;
}
*rsp = tmp2;
}
strcpy(*rsp, accum);//refresh rsp
}
}while(1);
}
// return '*str' after number of bytes realloc'ed to 'size'
static char * ReSizeBuffer(char **str, unsigned int size)
{
char *tmp=NULL;
if(!(*str)) return NULL;
if(size == 0)
{
free(*str);
return NULL;
}
tmp = (char *)realloc((char *)(*str), size);
if(!tmp)
{
free(*str);
return NULL;
}
*str = tmp;
return *str;
}
static void SetupSecurityAttributes(SECURITY_ATTRIBUTES *saAttr)
{
// Set the bInheritHandle flag so pipe handles are inherited.
saAttr->nLength = sizeof(SECURITY_ATTRIBUTES);
saAttr->bInheritHandle = TRUE;
saAttr->lpSecurityDescriptor = NULL;
}
static void SetupStartUpInfo(STARTUPINFO *siStartInfo, IO_PIPES *io)
{
siStartInfo->cb = sizeof(STARTUPINFO);
siStartInfo->hStdError = io->out_pipe_write;
siStartInfo->hStdOutput = io->out_pipe_write;
siStartInfo->hStdInput = io->in_pipe_read;
siStartInfo->dwFlags |= STARTF_USESTDHANDLES;
}
3 Answers 3
Handling embedded null bytes
ReadFile(, lpBuffer,,,)
may read null characters into lpBuffer
. Should this occur, much of code's use of str...()
would suffer. Code instead needs to keep track of data as a read of "bytes" with a length and not as a string. I'd recommend forming a structure with members unsigned char data[BUF_SIZE]
and DWORD sz
or size_t sz
. This affects code significantly. Effectively replace str...()
calls with mem...()
ones.
Minor: with using a "byte" buffer rather than a string, the buffer could start with NULL
.
// char *accum;
// accum = (char *)calloc(1, sizeof(char));
char *accum = NULL;
Pipe creation and usage
Although there is much good error handling, cmd_rsp()
fails to check the return value of ReadFromPipe(chunk, chunk_size, &io);
.
// ReadFromPipe(chunk, chunk_size, &io);
if (ReadFromPipe(chunk, chunk_size, &io) == -1) TBD_Code();
Minor: Using sizeof(char)
rather than sizeof *accum
obliges a reviewer and maintainer to check the type of accum
. In C, code can be simplified:
// accum = (char *)calloc(1, sizeof(char));
accum = calloc(1, sizeof *accum);
Minor: Unclear why code is using unsigned
for array indexing rather than the idiomatic size_t
. Later code quietly returns this as an int
. I'd expect more care changing sign-ness. Else just use int
.
// Hmmm
unsigned int total_size = 0;
int size_recv = 0;
CreateProcess usage
Memory Leak:
if(SetupChildIoPipes(&io, &saAttr) < 0) {
free(Command); // add
return -1;
}
Minor: no need for int
cast of a value in the size_t
range.
// int len = (int)strlen(command);
// Command = calloc(len + sizeof(rqdStr), 1);
Command = calloc(strlen(command) + sizeof(rqdStr), 1);
Method for dynamically growing response buffer
Good and proper function for ReSizeBuffer( ,size == 0);
Bug: when realloc()
fails, ReSizeBuffer()
and the calling code both free the same memory. Re-design idea: Let ReSizeBuffer()
free the data and return a simple fail/success flag for the calling code to test. For the calling code to test NULL
-ness is a problem as ReSizeBuffer( ,size == 0)
returning NULL
is O.K.
Unclear test: if(!(*str)) return NULL;
. I would not expect disallowing resizing a buffer that originally pointed to NULL
.
if(!(*str)) return NULL; // why?
if(!str) return NULL; // Was this wanted?`
Cast not needed for a C compile. Is code also meant for C++?
// tmp = (char *)realloc((char *)(*str), size);
tmp = realloc(*str, size);
For me, I would use the form below and let it handle all edge cases of zeros, overflow, allocation success, free-ing, updates. Be prepared for large buffer needs.
// return 0 on success
int ReSizeBuffer(void **buf, size_t *current_size, int increment);
// or
int ReSizeBuffer(void **buf, size_t *current_size, size_t new_size);
Tidbits
Consider avoiding !
when things work. This is a small style issue - I find a !
or !=
more aligns with failure than success.
// if (!cmd_no_rsp("dir /s", &buf, BUF_SIZE)) {
// printf("success.\n");
if (cmd_no_rsp("dir /s", &buf, BUF_SIZE) == 0) {
printf("success.\n");
With a change for handling piped data as a string, change variables names away from str...
Handling embedded null bytes in content returned from
ReadFile
function
you not need handle this at all. ReadFile
return number of bytes read. and you need use it. no any difference what bytes you read.
(*rsp)[dwRead] = 0;
size_recv = (int)strlen(*rsp);
this code wrong and senseless.
correct (by sense)
size_recv = dwRead
the dwRead
is exactly how many bytes you read (if ReadFile
not fail)
SetCommTimeouts
again - senseless code. this api simply send IOCTL_SERIAL_SET_TIMEOUTS
control code to device on which file open. usually only serial controller driver handle this ioctl. the npfs.sys, which implement pipes - not understand and not support this ioctl. driver return to you STATUS_INVALID_DEVICE_REQUEST
( mapped to ERROR_INVALID_FUNCTION
win32 error) but you not check for errors here.
Method for dynamically growing response buffer (very interested in feedback on this)
the code is appalling. completely wrong in everything. begin from interface declaration:
_Inout_ char **chunk, _In_ unsigned int chunk_size
here chunk
- in/out parameter, but chunk_size
in only - if you reallocate user buffer - you must return new buffer size to user. may be next signature:
_Inout_ char **chunk, _Inout_ unsigned int* chunk_size
then reallocate caller supplied buffer - this is very bad idea. for reallocate you need exactly know how caller allocate buffer. you must add to interface contract - how caller must allocate initial buffer and free final. which concrete routine use. say for example caller must allocate initial buffer with LocalAlloc(LMEM_FIXED, *)
and free with LocalFree
. but nobody do this. usual used 2 ways:
api use user buffer as is
_In_ char *chunk, _In_ SIZE_T chunk_size, _Out_ SIZE_T* ReturnSize
if buffer not big enough - error is returned -
ERROR_INSUFFICIENT_BUFFER
(in case no valid data in buffer at all) orERROR_BUFFER_OVERFLOW
- in case exist some valid data in buffer, but need large buffer anyway. anyway you need here additional out parameterReturnSize
allocate buffer yourself
_Out_ char** chunk, _Out_ SIZE_T* chunk_size
in this case
chunk
andchunk_size
out only parameters. and you need say to caller - which api need use for free returned buffer.
then look for ReadFromPipe
(full nightmare)
bSuccess = ReadFile(io->out_pipe_read, *rsp, size-1, &dwRead, NULL)
;
you all time read to size-1
to buffer begin (*rsp
) and never change size
. so what sense try reallocate buffer if you any way try read only size-1
bytes. then you all time read to buffer begin - so new read overwrite previous data. you use wrong and senseless strcpy
, strcat
, strlen
instead of memcpy
. for what you use free
before realloc
? when you try reallocate buffer after every read ? even if still exist free space in current buffer ? on which size you try realloc buffer ? on 1 byte ?? and every time anyway try read constant size-1
to begin ??
for dynamic buffer buffer usually used 2 strategy:
allocate memory chunk (usually 0x1000..0x10000 size). read to this chunk until exist free space in it (of course not all time to the begin of chunk but to the begin of free space inside chunk). when no more free space in current chunk - allocate new chunk (not reallocate existing !!) and so on. but not copy anything. when read will be completed - once allocate new buffer and once copy content of chunks to this buffer. for example.
struct DATA_BUFFER
{
struct DATA_CHUNK
{
DATA_CHUNK* _next;
ULONG _cbData;
BYTE _buf[0x10000];
DATA_CHUNK() : _next(0), _cbData(0) {}
};
DATA_CHUNK* _first, *_last;
DATA_BUFFER() : _first(0), _last(0) {}
~DATA_BUFFER()
{
if (DATA_CHUNK * chunk = _first)
{
do
{
PVOID pv = chunk;
chunk = chunk->_next;
delete pv;
} while (chunk);
}
}
PVOID AllocBuffer(PULONG pcb)
{
DATA_CHUNK* chunk = _last;
if (!chunk || chunk->_cbData == sizeof(chunk->_buf))
{
if (chunk = new DATA_CHUNK)
{
if (_first)
{
_last->_next = chunk;
_last = chunk;
}
else
{
_first = chunk;
_last = chunk;
}
}
else
{
return 0;
}
}
*pcb = sizeof(chunk->_buf) - chunk->_cbData;
return chunk->_buf + chunk->_cbData;
}
void AddData(ULONG cb)
{
_last->_cbData += cb;
}
PVOID GatherBuffer(PULONG pcb)
{
*pcb = 0;
if (DATA_CHUNK* chunk = _first)
{
ULONG cb = 0;
do
{
cb += chunk->_cbData;
} while (chunk = chunk->_next);
if (cb)
{
*pcb = cb;
if (PBYTE pb = new BYTE[cb])
{
PVOID pv = pb;
chunk = _first;
do
{
memcpy(pb, chunk->_buf, chunk->_cbData);
pb += chunk->_cbData;
PVOID p = chunk;
chunk = chunk->_next;
delete p;
} while (chunk);
_first = 0, _last = 0;
return pv;
}
}
}
return 0;
}
};
and possible usage
DATA_BUFFER db;
ULONG cb;
PVOID pv;
while (pv = db.AllocBuffer(&cb))
{
if (ReadFile(hFile, pv, cb, &cb, 0) && cb)
{
db.AddData(cb);
}
else break;
}
if (pv = db.GatherBuffer(&cb))
{
// use pv..
delete [] pv;
}
another way - use VirtualAlloc
for buffer. windows let reserve memory space. we can reserve tens of megabytes how minimum. this operation not allocate any memory but simply mark memory region as reserved. then we can already commit memory to region begin. when, after read, will be not enough committed memory - we commit more and so on. advantage - we from begin will be have contiguous memory - we never will be need copy/move/reallocate memory with this way. this solution is better when we assume big enough final data. example:
class DynamicBuffer
{
PBYTE _BaseAddress;
SIZE_T _dwReserve, _dwSize, _dwCommit;
static SIZE_T RoundSize(SIZE_T size)
{
static SIZE_T s_dwAllocationGranularity;
if (!s_dwAllocationGranularity)
{
SYSTEM_INFO si;
GetSystemInfo(&si);
s_dwAllocationGranularity = si.dwAllocationGranularity - 1;
}
return (size + s_dwAllocationGranularity) & ~s_dwAllocationGranularity;
}
public:
DynamicBuffer()
{
_BaseAddress = 0, _dwReserve = 0, _dwSize = 0, _dwCommit = 0;
}
~DynamicBuffer()
{
Reset();
}
ULONG Create(SIZE_T dwSize)
{
if (_BaseAddress = (PBYTE)VirtualAlloc(0, dwSize = RoundSize(dwSize), MEM_RESERVE, PAGE_READWRITE))
{
_dwReserve = dwSize;
return NOERROR;
}
return GetLastError();
}
ULONG AllocBuffer(PVOID* ppv, SIZE_T cb)
{
if (_dwReserve - _dwSize < cb)
{
return ERROR_OUTOFMEMORY;
}
SIZE_T dwSize = _dwSize + cb;
if (dwSize > _dwCommit)
{
SIZE_T dwCommit = RoundSize(dwSize);
if (!VirtualAlloc(_BaseAddress + _dwCommit, dwCommit - _dwCommit, MEM_COMMIT, PAGE_READWRITE))
{
return GetLastError();
}
_dwCommit = dwCommit;
}
*ppv = _BaseAddress + _dwSize;
return NOERROR;
}
void AddData(SIZE_T cb)
{
_dwSize += cb;
if (_dwSize > _dwCommit)
{
__debugbreak();
}
}
PVOID getData()
{
return _BaseAddress;
}
SIZE_T getDataSize()
{
return _dwSize;
}
SIZE_T getFreeSpace()
{
return _dwReserve - _dwSize;
}
void Reset()
{
if (_BaseAddress)
{
VirtualFree(_BaseAddress, 0, MEM_RELEASE);
_BaseAddress = 0;
}
_dwReserve = 0, _dwSize = 0, _dwCommit = 0;
}
};
and usage
DynamicBuffer db;
ULONG cb;
PVOID pv;
if (!db.Create(0x1000000))
{
while (!db.AllocBuffer(&pv, cb = 0x10000))
{
if (ReadFile(hFile, pv, cb, &cb, 0) && cb)
{
db.AddData(cb);
}
else break;
}
}
CloseHandle(hFile);
// use data
db.getData();
db.getDataSize();
// free in destructor
but main question - are you need Contiguous Memory buffer at all ? for what you need cmd output ? for send it to remote system, display it in interface ? for this not need. after you read some data from cmd in fixed size buffer - just display this chunk in interface or send to remote system. and begin read new data chunk. of course you need absolute another interface for this. for parsing cmd output ? for this yes - need Contiguous Memory buffer.. but we never need parse cmd output. for what ?! if say we want list files/folders - we need do this yourself but not run dir command
ansi function usage. such as CreateProcessA
this is very bad. windows is unicode (utf-8) system. almost all api implemented as unicode. the ansi (A) api - is shell over unicode (W) api. the A shell convert input ansi strings to unicode, call W api and finally convert out unicode strings to ansi. this is very not efficient. more bad that use ansi strings simply wrong by design. ansi code page is not invariant. it different on different systems. if you hardcode some ansi string in code, which use characters> 0x80 - you got different unicode strings after convert, on different systems. finally not any unicode string can be converted to the current ansi code page.
use ansi code page for cmd is wrong. cmd use for pipe input/output not ansi but oem code page. this is different code pages. when cmd read multi-byte string from stdin file - he convert it to unicode via MultiByteToWideChar
with CP_OEMCP
. and when he output something to stdout file - he translate unicode string to multi-byte via WideCharToMultiByte
with CP_OEMCP
. so until he output characters in range [0, 0x80)
you not view different. but if will be say "not english" file name in output, or you pass some command with "not english" file name - will be error - because you and cmd use different code pages for translation. you pass him ansi but he wait oem. he pass to you oem strings, but you wait ansi.
also note about inherited handles - begin from vista better use also UpdateProcThreadAttribute
with PROC_THREAD_ATTRIBUTE_HANDLE_LIST
for restrict list of handles to be inherited by the child process - not all inheritable handles, but only one pipe handle. yes one but not two as you use.
Pipe creation and usage
again bad and wrong. for what you create 2 pipe pairs ?? when 1 pipe pair only need. one pipe handle in you process and one connected pipe end in cmd. pipe must be duplex. what you write to this pipe end in self process - will be read in cmd. what cmd write to self handle - you read via handle in self process. so not need additional pipe pair. next - always need use asynchronous pipes and io here. synchronous is not efficient and can deadlock.
interface of course also must be absolute another. here faster need export class with virtual functions. class implement cmd exec and write commands to it. virtual callbacks with read data from cmd. you inherit own class from this base interface class, implement your own on read data handler and cmd exit (disconect). some basic implementation of class:
class CPacket
{
ULONG _BufferSize, _DataSize;
LONG _dwRef;
LONG _pad;
char _buf[];
enum ST : ULONG {};
~CPacket() {}
void* operator new(size_t ByteSize, ST BufferSize)
{
return ::operator new(ByteSize + BufferSize);
}
CPacket(ULONG BufferSize) : _dwRef(1), _BufferSize(BufferSize), _DataSize(0) { }
public:
CPacket() { }
void* operator new(size_t , ULONG BufferSize)
{
return new(static_cast<ST>(BufferSize)) CPacket(BufferSize);
}
void AddRef()
{
InterlockedIncrement(&_dwRef);
}
void Release()
{
if (!InterlockedDecrement(&_dwRef)) delete this;
}
PSTR getFreeBuffer()
{
return _buf + _DataSize;
}
ULONG getFreeSize()
{
return _BufferSize - _DataSize;
}
PSTR getData()
{
return _buf;
}
ULONG getBufferSize()
{
return _BufferSize;
}
ULONG getDataSize()
{
return _DataSize;
}
ULONG setDataSize(ULONG DataSize)
{
return _DataSize = DataSize;
}
};
struct U_IRP;
class __declspec(novtable) CCmd
{
friend U_IRP;
enum { read, write };
HANDLE _hFile;
CPacket* _packet;
LONG _dwRef, _handleLock;
ULONG Read();
static ULONG CreatePipeAnonymousPair(PHANDLE phServerPipe, PHANDLE phClientPipe);
static ULONG CreatePipeAnonymousPair7(PHANDLE phServerPipe, PHANDLE phClientPipe);
ULONG CreatePipePair(HANDLE* phClient);
void OnIoComplete(ULONG OpCode, ULONG dwErrorCode, ULONG_PTR dwNumberOfBytesTransfered, PSTR buf);
BOOL LockHandle();
void UnlockHandle();
protected:
virtual ~CCmd()
{
Close();
if (_packet)
{
_packet->Release();
}
}
virtual BOOL OnRead(PSTR buf, ULONG cb) = 0;
virtual void OnWrite(ULONG /*dwErrorCode*/) { }
public:
void Close();
ULONG Write(PCWSTR lpWideCharStr, int cchWideChar = -1);
ULONG Write(PCSTR szOem)
{
return Write(szOem, (ULONG)strlen(szOem));
}
ULONG Write(const void* pv, ULONG cb);
ULONG Write(CPacket* packet);
ULONG Exec(ULONG cbBuffer);
void AddRef()
{
InterlockedIncrement(&_dwRef);
}
void Release()
{
if (!InterlockedDecrement(&_dwRef))
{
delete this;
}
}
CCmd() : _hFile(0), _packet(0), _dwRef(1), _handleLock(0)
{
}
};
struct U_IRP : OVERLAPPED
{
CCmd* _pIoObject;
CPacket* _packet;
PSTR _pv;
ULONG _code;
~U_IRP()
{
if (_packet)
{
_packet->Release();
}
_pIoObject->Release();
}
ULONG CheckIoResult(BOOL fOk)
{
if (fOk)
{
OnIoComplete(NOERROR, InternalHigh);
}
else
{
ULONG dwErrorCode = GetLastError();
if (dwErrorCode != ERROR_IO_PENDING)
{
OnIoComplete(dwErrorCode, 0);
}
}
return NOERROR;
}
VOID OnIoComplete(ULONG dwErrorCode, ULONG_PTR dwNumberOfBytesTransfered)
{
_pIoObject->OnIoComplete(_code, dwErrorCode, dwNumberOfBytesTransfered, _pv);
delete this;
}
static VOID WINAPI _onIoComplete(
ULONG dwErrorCode,
ULONG dwNumberOfBytesTransfered,
LPOVERLAPPED lpOverlapped
)
{
static_cast<U_IRP*>(lpOverlapped)->OnIoComplete(RtlNtStatusToDosError(dwErrorCode), dwNumberOfBytesTransfered);
}
public:
U_IRP(CCmd* pIoObject, ULONG code = 0, CPacket* packet = 0, PSTR pv = 0) : _pIoObject(pIoObject), _code(code), _packet(packet), _pv(pv)
{
if (packet)
{
packet->AddRef();
}
pIoObject->AddRef();
RtlZeroMemory(static_cast<OVERLAPPED*>(this), sizeof(OVERLAPPED));
}
static ULONG Bind(HANDLE hFile)
{
return BindIoCompletionCallback(hFile, U_IRP::_onIoComplete, 0)
&& SetFileCompletionNotificationModes(hFile, FILE_SKIP_COMPLETION_PORT_ON_SUCCESS)
? NOERROR : GetLastError();
}
};
ULONG CCmd::Read()
{
ULONG err = ERROR_INVALID_HANDLE;
if (LockHandle())
{
err = ERROR_NO_SYSTEM_RESOURCES;
PSTR buf = _packet->getFreeBuffer();
if (U_IRP* Irp = new U_IRP(this, read, _packet, buf))
{
err = Irp->CheckIoResult(ReadFile(_hFile, buf, _packet->getFreeSize(), 0, Irp));
}
UnlockHandle();
}
return err;
}
ULONG CCmd::Write(const void* pv, ULONG cb)
{
if (CPacket* packet = new(cb) CPacket)
{
memcpy(packet->getData(), pv, packet->setDataSize(cb));
ULONG err = Write(packet);
packet->Release();
return err;
}
return ERROR_NO_SYSTEM_RESOURCES;
}
ULONG CCmd::Write(PCWSTR lpWideCharStr, int cchWideChar)
{
if (cchWideChar < 0)
{
cchWideChar = (ULONG)wcslen(lpWideCharStr);
}
if (int cbMultiByte = WideCharToMultiByte(CP_OEMCP, 0, lpWideCharStr, cchWideChar, 0, 0, 0, 0))
{
if (CPacket* packet = new(cbMultiByte) CPacket)
{
ULONG err;
if (cbMultiByte = WideCharToMultiByte(CP_OEMCP, 0, lpWideCharStr, cchWideChar, packet->getData(), cbMultiByte, 0, 0))
{
packet->setDataSize(cbMultiByte);
err = Write(packet);
}
else
{
err = GetLastError();
}
packet->Release();
return err;
}
return ERROR_NO_SYSTEM_RESOURCES;
}
return GetLastError();
}
ULONG CCmd::Write(CPacket* packet)
{
ULONG err = ERROR_INVALID_HANDLE;
if (LockHandle())
{
err = ERROR_NO_SYSTEM_RESOURCES;
if (U_IRP* Irp = new U_IRP(this, write, packet))
{
err = Irp->CheckIoResult(WriteFile(_hFile, packet->getData(), packet->getDataSize(), 0, Irp));
}
UnlockHandle();
}
return err;
}
ULONG CCmd::CreatePipeAnonymousPair(PHANDLE phServerPipe, PHANDLE phClientPipe)
{
static char flag_supported = -1;
if (flag_supported < 0)
{
ULONG dwMajorVersion, dwMinorVersion;
RtlGetNtVersionNumbers(&dwMajorVersion, &dwMinorVersion, 0);
flag_supported = _WIN32_WINNT_WIN7 <= ((dwMajorVersion << 8)| dwMinorVersion);
}
if (flag_supported)
{
return CreatePipeAnonymousPair7(phServerPipe, phClientPipe);
}
static LONG s;
if (!s)
{
ULONG seed = GetTickCount();
InterlockedCompareExchange(&s, RtlRandomEx(&seed), 0);
}
WCHAR name[64];
swprintf(name, L"\\\\?\\pipe\\Win32Pipes.%08x.%08x", GetCurrentProcessId(), InterlockedIncrement(&s));
HANDLE hClient, hServer = CreateNamedPipeW(name,
PIPE_ACCESS_DUPLEX|FILE_READ_DATA|FILE_WRITE_DATA|FILE_FLAG_OVERLAPPED,
PIPE_TYPE_BYTE|PIPE_READMODE_BYTE, 1, 0, 0, NMPWAIT_USE_DEFAULT_WAIT, 0);
if (hServer != INVALID_HANDLE_VALUE)
{
static SECURITY_ATTRIBUTES sa = { sizeof(sa), 0, TRUE };
hClient = CreateFileW(name, FILE_GENERIC_READ|FILE_GENERIC_WRITE,
FILE_SHARE_READ|FILE_SHARE_WRITE, &sa, OPEN_EXISTING, 0, 0);
if (hClient != INVALID_HANDLE_VALUE)
{
*phServerPipe = hServer, *phClientPipe = hClient;
return NOERROR;
}
CloseHandle(hServer);
}
return GetLastError();
}
ULONG CCmd::CreatePipeAnonymousPair7(PHANDLE phServerPipe, PHANDLE phClientPipe)
{
HANDLE hNamedPipe;
IO_STATUS_BLOCK iosb;
static UNICODE_STRING NamedPipe = RTL_CONSTANT_STRING(L"\\Device\\NamedPipe\\");
OBJECT_ATTRIBUTES oa = { sizeof(oa), 0, (PUNICODE_STRING)&NamedPipe, OBJ_CASE_INSENSITIVE };
NTSTATUS status;
if (0 <= (status = NtOpenFile(&hNamedPipe, SYNCHRONIZE, &oa, &iosb, FILE_SHARE_VALID_FLAGS, 0)))
{
oa.RootDirectory = hNamedPipe;
static LARGE_INTEGER timeout = { 0, MINLONG };
static UNICODE_STRING empty = {};
oa.ObjectName = ∅
if (0 <= (status = ZwCreateNamedPipeFile(phServerPipe,
FILE_READ_ATTRIBUTES|FILE_READ_DATA|
FILE_WRITE_ATTRIBUTES|FILE_WRITE_DATA|
FILE_CREATE_PIPE_INSTANCE,
&oa, &iosb, FILE_SHARE_READ|FILE_SHARE_WRITE,
FILE_CREATE, 0, FILE_PIPE_BYTE_STREAM_TYPE, FILE_PIPE_BYTE_STREAM_MODE,
FILE_PIPE_QUEUE_OPERATION, 1, 0, 0, &timeout)))
{
oa.RootDirectory = *phServerPipe;
oa.Attributes = OBJ_CASE_INSENSITIVE|OBJ_INHERIT;
if (0 > (status = NtOpenFile(phClientPipe, SYNCHRONIZE|FILE_READ_ATTRIBUTES|FILE_READ_DATA|
FILE_WRITE_ATTRIBUTES|FILE_WRITE_DATA, &oa, &iosb,
FILE_SHARE_VALID_FLAGS, FILE_SYNCHRONOUS_IO_NONALERT)))
{
NtClose(oa.RootDirectory);
}
}
NtClose(hNamedPipe);
}
return RtlNtStatusToDosError(status);
}
ULONG CCmd::CreatePipePair(HANDLE* phClient)
{
HANDLE hServer, hClient;
ULONG err = CreatePipeAnonymousPair(&hServer, &hClient);
if (!err)
{
_hFile = hServer;
_handleLock = 0x80000000;
if (!(err = U_IRP::Bind(hServer)) && !(err = Read()))
{
*phClient = hClient;
return NOERROR;
}
CloseHandle(hClient);
}
return err;
}
void CCmd::OnIoComplete(ULONG OpCode, ULONG dwErrorCode, ULONG_PTR dwNumberOfBytesTransfered, PSTR buf)
{
switch (OpCode)
{
case read:
if (dwErrorCode == NOERROR)
{
if (OnRead(buf, (ULONG)dwNumberOfBytesTransfered))
{
Read();
}
}
break;
case write:
OnWrite(dwErrorCode);
break;
default:
__debugbreak();
}
if (dwErrorCode)
{
DbgPrint("[%u]: error=%u\n", OpCode, dwErrorCode);
}
}
void CCmd::UnlockHandle()
{
if (!_InterlockedDecrement(&_handleLock))
{
CloseHandle(_hFile), _hFile = 0;
}
}
BOOL CCmd::LockHandle()
{
LONG Value, NewValue;
if (0 > (Value = _handleLock))
{
do
{
NewValue = _InterlockedCompareExchange(&_handleLock, Value + 1, Value);
if (NewValue == Value) return TRUE;
} while (0 > (Value = NewValue));
}
return FALSE;
}
void CCmd::Close()
{
if (LockHandle())
{
_bittestandreset(&_handleLock, 31);
UnlockHandle();
}
}
ULONG CCmd::Exec(ULONG cbBuffer)
{
WCHAR ApplicationName[MAX_PATH];
if (!GetEnvironmentVariableW(L"ComSpec", ApplicationName, RTL_NUMBER_OF(ApplicationName)))
{
return GetLastError();
}
if (!(_packet = new(cbBuffer) CPacket))
{
return ERROR_NO_SYSTEM_RESOURCES;
}
STARTUPINFOEXW si = { { sizeof(si) } };
PROCESS_INFORMATION pi;
ULONG err = CreatePipePair(&si.StartupInfo.hStdError);
if (err)
{
return err;
}
si.StartupInfo.dwFlags = STARTF_USESTDHANDLES;
si.StartupInfo.hStdInput = si.StartupInfo.hStdOutput = si.StartupInfo.hStdError;
ULONG dwCreationFlags = CREATE_NO_WINDOW;
BOOL fInit = FALSE;
SIZE_T Size;
if (!InitializeProcThreadAttributeList(0, 1, 0, &Size) &&
GetLastError() == ERROR_INSUFFICIENT_BUFFER &&
InitializeProcThreadAttributeList(si.lpAttributeList = (LPPROC_THREAD_ATTRIBUTE_LIST)alloca(Size), 1, 0, &Size))
{
fInit = TRUE;
if (UpdateProcThreadAttribute(si.lpAttributeList, 0, PROC_THREAD_ATTRIBUTE_HANDLE_LIST,
&si.StartupInfo.hStdError, sizeof(HANDLE), 0, 0))
{
dwCreationFlags |= EXTENDED_STARTUPINFO_PRESENT;
}
}
if (CreateProcessW(ApplicationName, 0, 0, 0, TRUE, dwCreationFlags, 0, 0, &si.StartupInfo, &pi))
{
CloseHandle(pi.hThread);
CloseHandle(pi.hProcess);
}
else
{
err = GetLastError();
}
if (fInit)
{
DeleteProcThreadAttributeList(si.lpAttributeList);
}
CloseHandle(si.StartupInfo.hStdError);
return err;
}
and usage example
class CMyCmd : public CCmd
{
ULONG _dwThreadId;
virtual ~CMyCmd()
{
PostThreadMessage(_dwThreadId, WM_QUIT, 0, 0);
}
static void PrintOem(PSTR buf, ULONG cb)
{
if (int cchWideChar = MultiByteToWideChar(CP_OEMCP, 0, buf, cb, 0, 0))
{
PWSTR wz = (PWSTR)alloca(cchWideChar * sizeof(WCHAR));
if (MultiByteToWideChar(CP_OEMCP, 0, buf, cb, wz, cchWideChar))
{
if (ULONG cbMultiByte = WideCharToMultiByte(CP_ACP, 0, wz, cchWideChar, 0, 0, 0, 0))
{
PSTR sz = (PSTR)alloca(cbMultiByte);
if (WideCharToMultiByte(CP_ACP, 0, wz, cchWideChar, sz, cbMultiByte, 0, 0))
{
DbgPrint("%.*s", cbMultiByte, sz);
}
}
}
}
}
virtual BOOL OnRead(PSTR buf, ULONG cbTotal)
{
if (cbTotal)
{
ULONG cb;
do
{
PrintOem(buf, cb= min(cbTotal, 256));
} while (buf += cb, cbTotal -= cb);
}
return TRUE;
}
public:
CMyCmd() : _dwThreadId(GetCurrentThreadId()) {}
};
void CmdTest()
{
if (CCmd* pCmd = new CMyCmd)
{
if (!pCmd->Exec(0x1000))
{
pCmd->Write("echo off\r\n");
pCmd->Write(L"echo ***\r\n");
pCmd->Write("cd /d d:\\\r\ndir\r\nexit\r\n");
}
pCmd->Release();
}
MessageBoxW(0,L"demo user interface",L"wait signal",0);
}
-
\$\begingroup\$ @ryyker - yes, i view c tag. but i not use any c++ std libs. code of course can be transformed to c easy. simply it will be bit bigger and less convenient. really i focused not on language but on winapi and windows prorgamming \$\endgroup\$RbMm– RbMm2018年03月13日 12:50:26 +00:00Commented Mar 13, 2018 at 12:50
-
\$\begingroup\$ @ryyker - your error exactly in use string functions to manipulate buffer. and i try explain it at begin. when
ReadFile
is finished - you got exactly byte size you read. usestrlen
for this - error and for what - you already have size \$\endgroup\$RbMm– RbMm2018年03月13日 13:22:09 +00:00Commented Mar 13, 2018 at 13:22 -
\$\begingroup\$ @ryyker - what different will be 0 byte in buffer or not ? absolute no different for read file and buffer manipulation. not need care about this at all \$\endgroup\$RbMm– RbMm2018年03月13日 13:23:11 +00:00Commented Mar 13, 2018 at 13:23
-
\$\begingroup\$ I am not sure I follow. The primary purpose of the library is to return a string buffer to my calling application. If there is an embedded Null byte somewhere in the accumulation buffer, then will it not return incorrect results? i.e. I eventually have to use string functions to pull the results buffer from the function argument (accumulation buffer) into my application. \$\endgroup\$ryyker– ryyker2018年03月13日 13:28:24 +00:00Commented Mar 13, 2018 at 13:28
-
\$\begingroup\$ @ryyker - return a string buffer - you can return only what
ReadFile
return to you. as is. you can interpret this as string or no. this is already another question - how caller will be use returned buffer. also note - that cmd return not ansi, but oem encoding string. until say only english file names indir
output - no different. but on first "not english" name/word - you view different between ansi and oem. also i think you must not return single buffer to caller synchronous - this is wrong design. you need call caller callback with partial read data and size \$\endgroup\$RbMm– RbMm2018年03月13日 13:37:29 +00:00Commented Mar 13, 2018 at 13:37
You can call a builtin function of c in this way:
system("notepad");
-
2\$\begingroup\$ Please explain how this meets the requirements of the problem. It doesn't do quite the same task. In particular, the output of the program is handled differently. \$\endgroup\$200_success– 200_success2018年03月02日 02:37:16 +00:00Commented Mar 2, 2018 at 2:37
-
\$\begingroup\$ It is just an example, you can use scanf for dynamic inputs and pass to the function to exectue for OS. It is something related to Common Sence. \$\endgroup\$Ashfaq– Ashfaq2018年03月02日 03:31:10 +00:00Commented Mar 2, 2018 at 3:31
ReadFile(, lpBuffer,,,)
, AFAIK, may read null characters intolpBuffer
. Should this occur, it looks like much of code's use ofstr...()
would suffer. What is the design goal concerning piped null characters? \$\endgroup\$ReadFile()
and family operate in binary mode, so picking up anull
character could happen as part of normal data transfer, and indeed would affect how the string functions would interpret the data. And it would be a problem if the 1st character wasnull
. The honest answer about design goals for pipednull
characters is that I didn't even consider it. Nice catch. Any ideas? \$\endgroup\$ReadFromPipe
function to exclude the use of any string functions, or handle anynull
character detection in-line as the data is read. Perhaps something like THIS . \$\endgroup\$