Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

PHPORM-215 Implement Schema::getColumns and getIndexes #3045

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
GromNaN merged 4 commits into mongodb:4.7 from GromNaN:PHPORM-215
Jul 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ All notable changes to this project will be documented in this file.
## [4.9.0] - coming soon

* Add `Connection::getServerVersion()` by @GromNaN in [#3043](https://github.com/mongodb/laravel-mongodb/pull/3043)
* Add `Schema\Builder::getTables()` and `getTableListing` by @GromNaN in [#3044](https://github.com/mongodb/laravel-mongodb/pull/3044)
* Add `Schema\Builder::getTables()` and `getTableListing()` by @GromNaN in [#3044](https://github.com/mongodb/laravel-mongodb/pull/3044)
* Add `Schema\Builder::getColumns()` and `getIndexes()` by @GromNaN in [#3045](https://github.com/mongodb/laravel-mongodb/pull/3045)

## [4.6.0] - 2024年07月09日

Expand Down
82 changes: 82 additions & 0 deletions src/Schema/Builder.php
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,16 @@

use Closure;
use MongoDB\Model\CollectionInfo;
use MongoDB\Model\IndexInfo;

use function array_keys;
use function assert;
use function count;
use function current;
use function implode;
use function iterator_to_array;
use function sort;
use function sprintf;
use function usort;

class Builder extends \Illuminate\Database\Schema\Builder
Expand Down Expand Up @@ -146,6 +151,83 @@ public function getTableListing()
return $collections;
}

public function getColumns($table)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method could get slow for larger collections, so it may be worth adding a $sample pipeline stage down the road.

Depending on how much the feature is used, we can also consider expanding its functionality in a separate project to be more similar to Compass' schema analysis feature. For now, the current solution provides a good starting point.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should try to replicate what is done by Compass. I added $sample and $limit stages.

alcaeus reacted with thumbs up emoji
Copy link
Member Author

@GromNaN GromNaN Jul 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried on atlas sample dataset sample_airbnb.listingsAndReviews. With $sample, I get this error:

 PlanExecutor error during aggregation :: caused by :: Sort exceeded memory limit of 33554432 bytes, but did not opt in to external sorting. Aborting operation. Pass allowDiskUse:true to opt in.

It works without it. There might be an optimization on the server for this use-case.

{
$stats = $this->connection->getMongoDB()->selectCollection($table)->aggregate([
// Sample 1,000 documents to get a representative sample of the collection
['$sample' => ['size' => 1_000]],
// Convert each document to an array of fields
['$project' => ['fields' => ['$objectToArray' => '$$ROOT']]],
// Unwind to get one document per field
['$unwind' => '$fields'],
// Group by field name, count the number of occurrences and get the types
[
'$group' => [
'_id' => '$fields.k',
'total' => ['$sum' => 1],
'types' => ['$addToSet' => ['$type' => '$fields.v']],
],
],
// Get the most seen field names
['$sort' => ['total' => -1]],
// Limit to 1,000 fields
['$limit' => 1_000],
// Sort by field name
['$sort' => ['_id' => 1]],
], [
'typeMap' => ['array' => 'array'],
'allowDiskUse' => true,
])->toArray();

$columns = [];
foreach ($stats as $stat) {
sort($stat->types);
$type = implode(', ', $stat->types);
$columns[] = [
'name' => $stat->_id,
'type_name' => $type,
'type' => $type,
'collation' => null,
'nullable' => $stat->_id !== '_id',
'default' => null,
'auto_increment' => false,
'comment' => sprintf('%d occurrences', $stat->total),
'generation' => $stat->_id === '_id' ? ['type' => 'objectId', 'expression' => null] : null,
];
}

return $columns;
}

public function getIndexes($table)
{
$indexes = $this->connection->getMongoDB()->selectCollection($table)->listIndexes();

$indexList = [];
foreach ($indexes as $index) {
assert($index instanceof IndexInfo);
$indexList[] = [
'name' => $index->getName(),
'columns' => array_keys($index->getKey()),
'primary' => $index->getKey() === ['_id' => 1],
'type' => match (true) {
$index->isText() => 'text',
$index->is2dSphere() => '2dsphere',
$index->isTtl() => 'ttl',
default => 'default',
},
'unique' => $index->isUnique(),
];
}

return $indexList;
}

public function getForeignKeys($table)
{
return [];
}

/** @inheritdoc */
protected function createBlueprint($table, ?Closure $callback = null)
{
Expand Down
67 changes: 67 additions & 0 deletions tests/SchemaTest.php
View file Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,11 @@

use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Schema;
use MongoDB\BSON\Binary;
use MongoDB\BSON\UTCDateTime;
use MongoDB\Laravel\Schema\Blueprint;

use function collect;
use function count;

class SchemaTest extends TestCase
Expand Down Expand Up @@ -416,6 +419,70 @@ public function testGetTableListing()
$this->assertContains('newcollection_two', $tables);
}

public function testGetColumns()
{
$collection = DB::connection('mongodb')->collection('newcollection');
$collection->insert(['text' => 'value', 'mixed' => ['key' => 'value']]);
$collection->insert(['date' => new UTCDateTime(), 'binary' => new Binary('binary'), 'mixed' => true]);

$columns = Schema::getColumns('newcollection');
$this->assertIsArray($columns);
$this->assertCount(5, $columns);

$columns = collect($columns)->keyBy('name');

$columns->each(function ($column) {
$this->assertIsString($column['name']);
$this->assertEquals($column['type'], $column['type_name']);
$this->assertNull($column['collation']);
$this->assertIsBool($column['nullable']);
$this->assertNull($column['default']);
$this->assertFalse($column['auto_increment']);
$this->assertIsString($column['comment']);
});

$this->assertEquals('objectId', $columns->get('_id')['type']);
$this->assertEquals('objectId', $columns->get('_id')['generation']['type']);
$this->assertNull($columns->get('text')['generation']);
$this->assertEquals('string', $columns->get('text')['type']);
$this->assertEquals('date', $columns->get('date')['type']);
$this->assertEquals('binData', $columns->get('binary')['type']);
$this->assertEquals('bool, object', $columns->get('mixed')['type']);
$this->assertEquals('2 occurrences', $columns->get('mixed')['comment']);

// Non-existent collection
$columns = Schema::getColumns('missing');
$this->assertSame([], $columns);
}

public function testGetIndexes()
{
Schema::create('newcollection', function (Blueprint $collection) {
$collection->index('mykey1');
$collection->string('mykey2')->unique('unique_index');
$collection->string('mykey3')->index();
});
$indexes = Schema::getIndexes('newcollection');
$this->assertIsArray($indexes);
$this->assertCount(4, $indexes);

$indexes = collect($indexes)->keyBy('name');

$indexes->each(function ($index) {
$this->assertIsString($index['name']);
$this->assertIsString($index['type']);
$this->assertIsArray($index['columns']);
$this->assertIsBool($index['unique']);
$this->assertIsBool($index['primary']);
});
$this->assertTrue($indexes->get('_id_')['primary']);
$this->assertTrue($indexes->get('unique_index_1')['unique']);

// Non-existent collection
$indexes = Schema::getIndexes('missing');
$this->assertSame([], $indexes);
}

protected function getIndex(string $collection, string $name)
{
$collection = DB::getCollection($collection);
Expand Down

AltStyle によって変換されたページ (->オリジナル) /